Impressions of ChatGPT Agent + o3-Pro

Can something be 'meh' and 'wow' at the same time?

Jul 18, 2025

ChatGPT Agent

My first impression of ChatGPT Agents wasn't great, but I'm still optimistic. When used with o3 follow-ups you can cover a lot of ground very quickly.

TOPIC: I uploaded a review of the immuno-oncology space and asked it to come up with an innovative startup idea that is under-represented. I said I was particularly interested in ideas that would make attractive acquisition targets for larger pharma companies.
RESPONSE: Targeted Ferroptosis Inducers to Boost Immunotherapy
EXECUTION TIME: 7 mins
REPORT WORD COUNT: 1094
CITATIONS: 1 ← ????

I then asked the Agent to create a slide deck showing the business opportunity, competitive landscape, related research, and resource needs. Here's an image of one of the slides. It doesn't scream super-intelligence to me, but here are the slides so you can judge for yourself. Note that creating the slide deck took three tries and about 30 mins for the Agent. So...yea....not going to use it to make slides anytime soon.

Critical Analysis of Agent Output with o3-Pro

As a next step, I exported the report as a pdf and re-imported it into ChatGPT. I turned off Agent and set the model to o3-pro (OpenAI's flagship reasoning model....yes I pay $200/mo....no, I'm not sure it's worth it)

Prompt:
Perform a critical review and SWOT analysis of the opportunity presented in this document. Research recent developments in the field. I want to know:
1 - How novel is the idea?
2 - How feasible is the approach?
3 - How big is the market (be realistic, I'm not asking about the total cancer market)?
4 - What companies are already in this space that could be either competitors or potentially interested in acquiring an asset we develop?
5 - What is the regulatory path and are there any obvious hurdles?

o3-pro was very impressive. It spent 6 mins reading the doc and doing its own research. Over 120 sources were cited but it's unclear how many were actually read end-to-end. Given that many sources were from pubmed and behind paywalls, I suspect it didn't read past the abstract.

Here are a couple of bits from the report. You can see it in full, here.

Novelty of the idea

Mechanistic: Moderately novel. Ferroptosis itself has been recognised for a decade, but clinical translation—especially tumour‑targeted nanoparticles + anti‑PD‑(L)1—remains largely pre‑clinical.

Competitive landscape: Sparse but emerging. Only ~10 private or small‑cap firms globally list ferroptosis as a core modality (e.g., Prothegen, MitoImmune, PTC Therapeutics, FerroptoCure, SeaBeLife, FELIQS).

Recent academic momentum: Surge of 2024‑25 reviews and animal data showing checkpoint‑synergy, nanoparticle delivery and biomarker strategies.

Industry setbacks: Kojin TX’s wind‑down (Feb 2025) underscores translational and funding headwinds, but also leaves IP/ talent available.

Bottom line: Still novel enough for differentiated positioning, but no longer first‑in‑class; investors will demand convincing proof‑of‑concept.

SWOT Analysis

Recommendations

Kill the PK/Tox risk early – run parallel medicinal‑chemistry and NP‑engineering to hit ≥2 h plasma half‑life and tumour‑to‑liver ratio >5:1 before IND.
Secure a checkpoint‑inhibitor partner by IND filing to share safety database and enable cross‑labeling.
Lock biomarker strategy now – start retrospective analyses of SLC7A11/GPX4 expression in accessible tumour banks to justify inclusion criteria.
Stage‑gate financing – raise a modest seed to reach in‑vivo PoC; leverage non‑dilutive orphan disease grants; plan Series A only after demonstrable tumour regression in dual‑therapy models.
IP fencing – file composition‑of‑matter on novel GPX4/system x_c⁻ inhibitors and NP surface ligands; monitor abandoned Kojin applications for acquisition.

Final Thoughts

To restate my first point, ChatGPT Agent is not going to magically solve the world's problems or cause jobs to vanish. But these tools are remarkable time-savers and a great way to explore ideas.

General Approach

Figuring out when to use which model or tool took me some time, but this is the basic flow that has shaped up for me. It’s much faster than Google/click/read/Google/click/read but could still be better.

A flowchart diagram illustrating the stages of an AI-assisted decision-making process. The horizontal flow includes five blue-labeled boxes connected by arrows: idea (actor: human) → idea expansion (actor: LLM, e.g., GPT-4o, Claude) → research & report (actor: Agent, Deep Research, or reasoning model (e.g., o3) with web search) → critical analysis (actor: Reasoning model (e.g., o3) with web search) → review & decision (actor: human). Each stage has a designated actor, shown below the stages in purple text, indicating whether a human, an LLM, or an AI agent is responsible. — General Flow for AI-Augmented Decision Support

What I’d Like to See Next

Research isn’t a linear process, it’s a graph. IMO, these tools need to support branched research that allows the user to decide which paths to explore further. Actually, this would be pretty easy to vibe code. Hmmm…..

A concept map visualizing the connections between core tasks in data analysis and benchmarking LLM agents. The central nodes are “Data Analysis and Interpretation” (orange) and “Benchmarking LLM Agents” (blue), each connected to related topics. “Data Analysis and Interpretation” links to: “Statistical Modeling and Hypothesis Testing” (green) “Data Cleaning and Preprocessing” (green) “Exploratory Data Analysis (EDA)” (green) “Defining Evaluation Metrics” (orange) “Selecting Benchmark Tasks” (orange) “Benchmarking LLM Agents” (blue) “Benchmarking LLM Agents” links to: “LLM Agent Benchmarking” (black) “Experimental Setup and Design” (orange) “Statistical Modeling and Hypothesis Testing” further connects to: “Classification Models” (red) “Regression Analysis” (red) “Results Interpretation and Reporting” (green) Each node is color-coded by category, suggesting different themes (e.g. modeling, data preparation, evaluation, benchmarking). The diagram illustrates how foundational data science skills support effective LLM benchmarking. — Research Graph

Don't over(look|state) the obvious

Discussion about this post