Skip to main content

OpenAI Agents SDK integration

Copy for LLM

The OpenAI Agents Python SDK enables you to build AI agents powered by OpenAI's language models that can use tools, manage context, and interact with external systems through the Model Context Protocol (MCP). By connecting to the Apify MCP server, your agents can access Apify's extensive library of Actors to perform web scraping, data extraction, and automation tasks in real time.

For more details about the OpenAI Agents SDK, see the official documentation.

Prerequisites

Before integrating Apify with OpenAI Agents SDK, you'll need:

  • An Apify account - Sign up for an Apify account, if you don't have one.

  • Apify API token - Get your API token from the Integrations section in Apify Console. This token authorizes the MCP server to run Actors on your behalf. Make sure to keep it secure.

  • OpenAI API key - Get your API key from the OpenAI platform. You need this to use OpenAI Agents SDK.

  • Python packages - Install the required packages:

    pip install agents openai

Building a web search agent with Apify MCP

You can connect to the Apify MCP server using streamable HTTP with Bearer token authentication. Use your Apify API token by setting the Authorization: Bearer <APIFY_TOKEN> header in the MCP server configuration.

import asyncio
import os

from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp

os.environ["APIFY_TOKEN"] = "Your Apify API token"
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"


async def main() -> None:
    # Create MCP server connection with Bearer token
    async with MCPServerStreamableHttp(
        name="Apify MCP Server",
        params={
            "url": "https://mcp.apify.com",
            "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
            "timeout": 120,
        },
        cache_tools_list=True,
        max_retry_attempts=3,
    ) as server:
        # Create agent with MCP server
        agent = Agent(
            name="Assistant",
            instructions="Use the Apify MCP tools to answer questions. Search the web when needed.",
            mcp_servers=[server],
        )

        # Run the agent
        result = await Runner.run(agent, "Search the web and summarize recent trends in AI agents")
        print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

Configuration options

The MCPServerStreamableHttp connects to the Apify MCP server using streamable HTTP. Key configuration options:

  • params: Dictionary containing:
    • url: The MCP server URL (https://mcp.apify.com for Apify)
    • headers: Authentication headers (Bearer token for Apify)
    • timeout: Request timeout in seconds
  • cache_tools_list: Caches the tool list to reduce API calls
  • max_retry_attempts: Number of retry attempts for failed requests
Tool execution may take some time

The agent may take some time to execute tool calls, especially when using web scraping Actors or searching the web. Actor runs can take time to complete depending on their task complexity.

Using specific Actors

You can configure the Apify MCP server to expose specific Actors by including them in the URL query parameters. For example, to use an Instagram scraper:

async with MCPServerStreamableHttp(
    name="Apify MCP Server",
    params={
        "url": "https://mcp.apify.com?tools=apify/instagram-scraper",
        "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
        "timeout": 120,
    },
    cache_tools_list=True,
) as server:
    agent = Agent(
        name="Assistant",
        instructions="Use the Instagram scraper to analyze Instagram profiles and posts.",
        mcp_servers=[server],
    )

    # Run the agent
    result = await Runner.run(
        agent, "Get the latest posts from @natgeo Instagram profile and summarize the content"
    )
    print(result.final_output)
Easy configuration

Use the UI configurator https://mcp.apify.com/ to select your tools visually, then copy the configuration to your code.

Examples

Web search agent

This example demonstrates how to build an agent that can search the web and use Apify Actors to gather information:

import asyncio
import os

from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp

os.environ["APIFY_TOKEN"] = "Your Apify API token"
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"


async def main() -> None:
    # Create MCP server connection
    async with MCPServerStreamableHttp(
        name="Apify MCP Server",
        params={
            "url": "https://mcp.apify.com",
            "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
            "timeout": 120,
        },
        cache_tools_list=True,
        max_retry_attempts=3,
    ) as server:
        # Create agent with MCP server
        agent = Agent(
            name="Assistant",
            instructions="Use the Apify MCP tools to answer questions. Search the web when needed.",
            mcp_servers=[server],
        )

        # Run the agent
        result = await Runner.run(agent, "Search the web and summarize recent trends in AI agents")
        print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

Instagram profile analysis

This example shows how to use a specific Actor for Instagram profile analysis:

import asyncio
import os

from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp

os.environ["APIFY_TOKEN"] = "Your Apify API token"
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"


async def main() -> None:
    # Create MCP server connection with Instagram scraper
    async with MCPServerStreamableHttp(
        name="Apify MCP Server",
        params={
            "url": "https://mcp.apify.com?tools=apify/instagram-scraper",
            "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
            "timeout": 120,
        },
        cache_tools_list=True,
        max_retry_attempts=3,
    ) as server:
        # Create agent with MCP server
        agent = Agent(
            name="Assistant",
            instructions="Use the Instagram scraper to analyze Instagram profiles and posts.",
            mcp_servers=[server],
        )

        # Run the agent
        result = await Runner.run(
            agent, "Get the latest posts from @natgeo Instagram profile and summarize the main themes"
        )
        print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

Agent to test MCP servers

You can use the OpenAI Agents SDK to test MCP servers and verify they're working correctly. The agent can list available tools and execute them to ensure proper functionality:

import asyncio
import os

from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp

os.environ["APIFY_TOKEN"] = "Your Apify API token"
os.environ["OPENAI_API_KEY"] = "Your OpenAI API key"


async def main() -> None:
    # Connect to Apify MCP server for testing
    async with MCPServerStreamableHttp(
        name="Apify MCP Server",
        params={
            "url": "https://mcp.apify.com",
            "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
            "timeout": 120,
        },
        cache_tools_list=True,
        max_retry_attempts=3,
    ) as server:
        # List available tools
        tools = await server.list_tools()
        print(f"Available tools: {[tool.name for tool in tools]}")

        # Create a test agent
        agent = Agent(
            name="Tester",
            instructions="Test the available MCP tools by calling them with appropriate parameters.",
            mcp_servers=[server],
        )

        # Test a simple query
        result = await Runner.run(
            agent, "List all available tools and test the search-actors tool with query 'instagram'"
        )
        print(result.final_output)


if __name__ == "__main__":
    asyncio.run(main())

For a comprehensive example with error handling and reporting, refer to the OpenAI Agent MCP Tester Actor. This Actor automates the testing process using OpenAI agents with a two-agent orchestrator pattern (planner and executor), generating detailed reports with pass/fail status for each tool.

Troubleshooting

Authentication errors

  • Check your API token: Verify that your Apify API token is correct. You can find it in the Integrations section of the Apify Console. Without a valid token, the server cannot start Actor runs.
  • Set environment variables: When running your agent, ensure you have set the APIFY_TOKEN and OPENAI_API_KEY environment variables.

Agent execution issues

  • No response or long delays: Actor runs can take time to complete depending on their task. If you're experiencing long delays, check the Actor's logs in Apify Console. The logs will provide insight into the Actor's status and show if it's processing a long operation or has encountered an error.
  • Tool execution timeout: If tool calls are timing out, increase the timeout parameter in the MCPServerStreamableHttp configuration.

Resources