--- question: "As of 2026, what emotion-aware, sentiment-aware, persona, memory, or affective features has OpenAI shipped or publicly announced across ChatGPT, the OpenAI API, Sora, Operator, and any custom GPTs / Agents SDK? Include the GPT-4o voice and emotion expressiveness, ChatGPT memory, Operator agent emotion handling, and any 2025 to 2026 announcements about emotional context. Does OpenAI expose any of this state through an API to third parties or only consume it internally? Cite working URLs." model: anthropic/claude-sonnet-4-6 max_output_tokens: 8000 asked_at: 2026-05-15T01:02:55.032Z total_tokens: 7012 status: completed cost_total_usd: 0.06734 cost_input_usd: 0.00002 cost_output_usd: 0.04806 cost_tools_usd: 0.005 --- # Q: As of 2026, what emotion-aware, sentiment-aware, persona, memory, or affective features has OpenAI shipped or publicly announced across ChatGPT, the OpenAI API, Sora, Operator, and any custom GPTs / Agents SDK? Include the GPT-4o voice and emotion expressiveness, ChatGPT memory, Operator agent emotion handling, and any 2025 to 2026 announcements about emotional context. Does OpenAI expose any of this state through an API to third parties or only consume it internally? Cite working URLs. # OpenAI Emotion-Aware, Sentiment, Persona, Memory & Affective Features — State of Play (May 2026) > **Scope:** ChatGPT (consumer app), OpenAI API / Realtime API, Sora, Operator, Agents SDK, Custom GPTs. Covers shipped features and public announcements from 2024 through May 2026. --- ## 1. GPT-4o Voice — Emotion Expressiveness ### What Was Shipped - **End-to-end audio model (launched May 2024):** GPT-4o was the first OpenAI model trained end-to-end across text, vision, and audio in a single neural network. This was a deliberate architectural departure — previous systems (e.g., the GPT-4 + Whisper pipeline) *lost* affective information because audio was transcribed to text before being processed. - **GPT-4o can now directly observe and output:** - Tone and prosody - Laughter and singing - Emotional expressiveness (e.g., excited, calm, empathetic delivery) - Background noise context - Multiple speaker dynamics - **Emotive style range:** Live demos at GPT-4o's launch showed the model generating voice in a "variety of different emotive styles" with a "wide dynamic range" — including whispering, dramatic narration, and enthusiastic registers. ([YouTube demo, May 2024](https://www.youtube.com/watch?v=D9byh4MAsUQ); [OpenAI announcement](https://openai.com/index/hello-gpt-4o/)) - **What was *not* shipped at launch:** OpenAI noted at the time that real-time emotional voice features were "still scratching the surface" and some capabilities (e.g., real-time emotion mirroring) were held back pending safety evaluation. ### API Exposure (Realtime API) - The **Realtime API** exposes GPT-4o audio capabilities to third-party developers, including emotional prosody in output voice. - As of December 2025, new model snapshots were shipped to the Realtime API: - `gpt-4o-mini-tts-2025-12-15` — 35% fewer word errors (Common Voice benchmark) - `gpt-realtime-mini-2025-12-15` — 22% improvement in instruction following, 13% better function calling - `gpt-4o-mini-transcribe-2025-12-15` — 89% reduction in hallucinations vs. Whisper-1 - ([OpenAI Community, Dec 2025](https://community.openai.com/t/new-audio-model-snapshots-in-the-realtime-api/1369374)) - **Third-party access:** ✅ Yes — developers can access expressive TTS, real-time audio in/out, and voice style modulation via the Realtime API and the newer TTS endpoints. --- ## 2. ChatGPT Memory ### What Was Shipped - **Persistent Memory (launched broadly 2024, significantly upgraded 2025–2026):** - ChatGPT can remember facts, preferences, and context across sessions. - Memory is **automatically managed** — as of the April 2026 release notes, ChatGPT now *automatically* prioritizes relevant memories and moves less important ones to the background, rather than requiring manual curation. ([OpenAI Help Center Release Notes, April 2026](https://help.openai.com/en/articles/6825453-chatgpt-release-notes)) - **Affective/persona relevance:** Memory can include emotionally relevant context — e.g., a user's communication style preferences, topics they find distressing, or tone preferences — enabling a form of *implicit persona modeling* across conversations. - **Projects feature (2025–2026):** ChatGPT's "Projects" feature groups conversations with persistent context and memory, allowing ongoing task continuity with emotional/stylistic state preserved. ### API Exposure - **Third-party access:** ⚠️ **Partially / Indirectly.** The OpenAI API does **not** expose ChatGPT's consumer memory store directly to third parties. Developers must implement their own memory layer (e.g., using vector databases, or the `previous_response_id` chaining in the Responses API). The **Agents SDK** and **Responses API** (extended in March 2026) support context compaction and reusable agent "skills," which are developer-managed analogs to memory — but this is not the same as the consumer ChatGPT memory system. ([InfoQ, March 2026](https://www.infoq.com/news/2026/03/openai-responses-api-agents/)) --- ## 3. Operator — Agent Emotional/Contextual Handling ### What Was Shipped - **Operator (launched January 2025, research preview for Pro users in the US):** An agent that controls a browser autonomously to complete tasks on behalf of users. ([OpenAI announcement](https://openai.com/index/introducing-operator/)) - Operator does **not** have publicly documented *explicit* emotion-detection or sentiment-routing features. However: - It inherits GPT-4o's general capability to interpret tone and intent from user instructions. - It uses **task context persistence** across agent steps, meaning user preferences and implicit emotional signals (e.g., urgency, frustration expressed in prompts) can influence task execution. - Safety guardrails include recognizing when a user appears distressed (linked to OpenAI's broader safety policies). ### API Exposure - **Third-party access:** ⚠️ **Limited.** Operator as a product is not directly API-accessible as an emotional-context layer. The underlying agentic infrastructure is exposed through the **Responses API + Agents SDK** (see Section 5), but emotional handling is not a discrete, documented API primitive. --- ## 4. Agents SDK & Responses API (2025–2026) ### What Was Shipped - The **Responses API** was extended in **March 2026** to support: - Built-in agent execution loops - A shell tool and hosted container workspace - **Context compaction** (managing long-running context windows — relevant to emotional/persona continuity) - **Reusable agent skills** — pre-packaged capability modules that agents can invoke - ([InfoQ, March 2026](https://www.infoq.com/news/2026/03/openai-responses-api-agents/)) - The **2025 developer retrospective** confirmed that 2025 was the year **agent-native APIs** became first-class, and multimodality (audio, image, video, docs) became standard API inputs/outputs. ([OpenAI for Developers in 2025](https://developers.openai.com/blog/openai-for-developers-2025)) ### Affective/Persona State via API - **System prompt persona:** Developers can inject persona definitions and emotional tone instructions via system prompts — this is fully supported and widely used. - **No discrete "emotion state" API object:** OpenAI does **not** expose a structured emotional state payload (e.g., `{"user_sentiment": "frustrated", "agent_affect": "empathetic"}`) through any public API endpoint. Emotional understanding is embedded in the model's latent reasoning, not surfaced as a queryable or subscribable state. - **Third-party access:** ✅ Tone/persona via system prompts; ✅ Expressive audio via Realtime API; ❌ No structured emotion-state API. --- ## 5. Custom GPTs ### What Was Shipped - **Custom GPTs** (GPT Builder, launched late 2023, iterated through 2025–2026) allow: - Custom **persona and tone instructions** baked into the system prompt - **Custom memory behaviors** (developers can configure what the GPT "remembers" or prioritizes) - **Affective style** defined via instructions (e.g., "always respond with warmth and encouragement") - These are consumer/builder-side configurations, not API-level emotional state primitives. ### API Exposure - **Third-party access:** ✅ Custom GPT behavior can be approximated via the API using system prompts and the Assistants API. The **Assistants API** supports persistent threads (a memory analog). However, the *ChatGPT-specific* persona/memory features of Custom GPTs are not programmable through the raw API in the same way. --- ## 6. Sora — Affective/Emotional Features ### What Was Shipped - **Sora** (video generation, launched December 2024) does not have documented emotion-aware or sentiment-aware *input processing* features in the sense of detecting user emotional state. - Sora can generate emotionally *evocative* video content based on descriptive prompts (e.g., "a melancholy scene," "joyful reunion"), meaning **affective language in prompts influences output mood/tone**. - No structured emotion API for Sora has been announced as of May 2026. --- ## 7. Summary Table | Feature | Product | Available to Users | Exposed via API to 3rd Parties | |---|---|---|---| | Expressive voice (emotion, tone, laughter) | ChatGPT / GPT-4o Audio | ✅ Yes | ✅ Yes (Realtime API, TTS API) | | Real-time audio emotion perception | GPT-4o Realtime | ✅ Yes | ✅ Yes (Realtime API) | | Persistent cross-session memory | ChatGPT (consumer) | ✅ Yes (auto-managed) | ❌ Not directly (dev must build own) | | Memory via Assistants/Responses API | API / Agents SDK | ✅ Yes (threads, context compaction) | ✅ Yes (developer-managed) | | Persona / tone via system prompt | API / Custom GPTs | ✅ Yes | ✅ Yes | | Structured emotion-state object in API | Any | ❌ Not shipped | ❌ Not available | | Operator emotional context | Operator agent | ⚠️ Implicit only | ❌ Not a discrete API feature | | Reusable agent skills (Responses API) | Agents SDK | ✅ Yes (March 2026) | ✅ Yes | | Sora affective prompt response | Sora | ✅ Yes (prompt-driven) | ⚠️ Limited API access | --- ## 8. Key Takeaway: Internal Consumption vs. Third-Party Exposure > **OpenAI primarily *consumes* affective/emotional state internally within its models rather than *exposing* it as a structured API primitive to third parties.** - The emotional intelligence in GPT-4o (tone detection, empathetic response, expressive voice) is baked into the model weights and surfaced through **natural language outputs and audio outputs** — not through structured emotion metadata fields. - The **Realtime API** is the closest thing to a "live emotional state" interface, as it streams raw audio in both directions and the model can respond to affective cues in real time. - No OpenAI API endpoint (as of May 2026) returns a JSON object like `{"detected_emotion": "anxious"}` or allows subscribing to user sentiment state — that inference happens inside the model and shapes the response, invisibly. - Developers who want explicit sentiment/emotion classification must either prompt the model to self-report it or use a separate classification pass. --- ## Sources - [OpenAI — Hello GPT-4o (May 2024)](https://openai.com/index/hello-gpt-4o/) - [OpenAI — Introducing Operator (Jan 2025)](https://openai.com/index/introducing-operator/) - [OpenAI Help Center — ChatGPT Release Notes (Apr 2026)](https://help.openai.com/en/articles/6825453-chatgpt-release-notes) - [OpenAI Community — Realtime API Audio Model Snapshots (Dec 2025)](https://community.openai.com/t/new-audio-model-snapshots-in-the-realtime-api/1369374) - [InfoQ — OpenAI Extends Responses API for Agents (Mar 2026)](https://www.infoq.com/news/2026/03/openai-responses-api-agents/) - [OpenAI for Developers in 2025 Retrospective](https://developers.openai.com/blog/openai-for-developers-2025) - [YouTube — GPT-4o Live Emotive Voice Demo (May 2024)](https://www.youtube.com/watch?v=D9byh4MAsUQ) - [ChatGPT Features 2026 Overview — SuprMind](https://suprmind.ai/hub/chatgpt/features/) --- ## Sources 1. [Hello GPT-4o - OpenAI](https://openai.com/index/hello-gpt-4o/) 2. [ChatGPT Features 2026: Projects, Memory, Agent, Sora and More](https://suprmind.ai/hub/chatgpt/features/) 3. [Introducing Operator - OpenAI](https://openai.com/index/introducing-operator/) 4. [ChatGPT — Release Notes - OpenAI Help Center](https://help.openai.com/en/articles/6825453-chatgpt-release-notes) 5. [OpenAI for Developers in 2025](https://developers.openai.com/blog/openai-for-developers-2025) 6. [New Audio Model Snapshots in the Realtime-API - Community](https://community.openai.com/t/new-audio-model-snapshots-in-the-realtime-api/1369374) 7. [OpenAI Extends the Responses API to Serve as a Foundation for ...](https://www.infoq.com/news/2026/03/openai-responses-api-agents/) 8. [Live demo of GPT4-o voice variation - YouTube](https://www.youtube.com/watch?v=D9byh4MAsUQ)