A comma in aggFilters returns 5% of the trials, and filter.phase does not exist at all

Here is the request that cost me the most time this week. It asks ClinicalTrials.gov for cancer trials in phases 2, 3 and 4, with the three values written the way a comma-separated list is normally written:

$ curl -s "https://clinicaltrials.gov/api/v2/studies?query.cond=cancer&countTotal=true&pageSize=1&aggFilters=phase:2,phase:3,phase:4"
{ "totalCount": 2598, ... }

2,598 studies. The right answer is 51,889. The response was HTTP 200. There was no warning field, no message, and nothing anywhere in the body to suggest that two of the three values I had asked for were discarded before the search ran. Five per cent of the matching trials, handed over as a success.

Everything below was measured on 2026-09-13 by sending requests to the live API one at a time, unauthenticated, with countTotal=true and pageSize=1 so that only the count came back. The registry publishes new data daily, so the absolute counts will have moved by the time you read this; the ratios are the point.

The comma keeps the last value, and nothing else

The first thing to establish is a baseline, one value at a time, so there is no argument about what the union should be:

Read the third row again. Asking for phase 2 and phase 3 returns exactly the phase 3 number, to the study. The 39,769 phase 2 trials are not filtered, not intersected, not ranked lower — they were never part of the query. The fourth row does the same thing one step further along: the count is the phase 4 count on its own, and both of the values in front of it are gone.

So the rule is not "the comma means AND", and it is not "the comma is invalid". The rule is that within one facet, the comma makes every value except the last one disappear. That is the worst of the three possible behaviours, because AND would have produced a suspiciously small number and invalid would have produced a 400. Instead the API produces a plausible number.

2,598 out of 51,889 is 5.0 per cent. If those rows are going into a dashboard, an alerting rule, or a market-sizing slide, nothing about the shape of the result will tell anyone that ninety-five per cent of it is missing.

The space is the separator that means OR

The separator that actually produces a union is a space. URL-encoded, the working request looks like this:

$ curl -s "https://clinicaltrials.gov/api/v2/studies?query.cond=cancer&countTotal=true&pageSize=1&aggFilters=phase%3A2%203%204"
{ "totalCount": 51889, ... }

Note the shape of it: the facet name appears once, and the values follow it separated by spaces — phase:2 3 4, not phase:2 phase:3 phase:4. This is the part that is easy to get wrong even after you know the rule, because every other query language you have used wants the field name repeated or the values bracketed.

And the comma is not useless. Between different facets it is the correct separator and it behaves as AND, which is what you would expect. So the fully correct form for "interventional studies in phase 2 or 3" is:

aggFilters=studyType:int,phase:2 3

One string, two separators, two different meanings, and no explanation of the distinction anywhere in the response. In my own client this is no longer something a caller can get wrong: the input is a list like ["phase:2", "phase:3", "studyType:int"], and a single function groups it by facet, joins the values within a facet with a space and the facets with a comma, and emits phase:2 3,studyType:int. The grouping is about thirty lines, and it is the only place in the codebase allowed to build that parameter.

The parameter that most write-ups name is rejected outright

Before I found the space rule, I did what anyone would do and searched for how to filter this API by phase. Several of the pages that rank for that question show the same thing in an example URL: a parameter called filter.phase, taking values like PHASE3, and in a couple of places an explicit note that you can comma-separate them as PHASE1,PHASE2. Those pages include a hands-on tutorial, a tool wrapper published by a university lab, and a reference file shipped inside a skill pack for an LLM agent. I am not going to link them or name the authors, because the interesting part is not that someone was wrong — it is what the API does when you follow them.

$ curl -s -w "HTTP %{http_code}\n" "https://clinicaltrials.gov/api/v2/studies?query.cond=cancer&countTotal=true&pageSize=1&filter.phase=PHASE3"
`filter.phase` is unknown parameter
HTTP 400

That body is the entire body. Not a JSON object with an error key — a bare backtick-quoted string. I sent it four ways on 2026-09-13, with PHASE2, PHASE3, PHASE4 and with the comma-separated list PHASE2,PHASE3,PHASE4, and all four came back with the identical 400 and the identical one-line body.

It is worth ruling out the obvious alternative explanation, which is that the whole filter.* family has been renamed or retired. It has not. In the same session, with the same base URL and the same condition:

?query.cond=cancer&countTotal=true&pageSize=1&filter.overallStatus=RECRUITING
-> HTTP 200, totalCount 18,775

filter.overallStatus is fine. filter.phase specifically is not a parameter this API has. Whether it once was, or whether it was inferred from the shape of its neighbours and then copied forward, I cannot tell from the outside and I am not going to guess. What I can report is what the live endpoint answered on 2026-09-13.

The loud failure is the one you want

Put the two failures side by side and the asymmetry is the whole lesson:

• filter.phase=PHASE2,PHASE3,PHASE4 — HTTP 400, no rows, immediately. You find this in the first minute of development and you never ship it.
• aggFilters=phase:2,phase:3,phase:4 — HTTP 200, 2,598 rows, forever. You ship this, it runs nightly, and the number it produces is stable, plausible and wrong.

The second one is genuinely more expensive than the first, and the ordering is not intuitive: the parameter that does not exist is safer to use than the parameter that does. A 400 with an unhelpful body still routes the problem to the right person. A 200 with a plausible count routes it to nobody.

This is also why "my tests are green" is not evidence here. A test that asserts the request was built correctly asserts your own belief about the syntax. A test that asserts the response parsed asserts that 2,598 is an integer. Neither one can see the gap. The only assertion that catches it is a count comparison: run each value on its own, run them together, and check that the combined count is at least the largest of the individual ones. Had I written that one assertion, it would have failed on the first run, because 10,997 is not greater than or equal to 39,769.

Why this particular silence turns into a bill

Undercounting is only half of the exposure. The same facet string is what stands between you and the entire registry. On 2026-09-10 I measured what happens when every filter is left blank: the API answers with a totalCount of 602,104, which is every study it holds. It does not take an empty query string to get there — one unset variable interpolated into a URL, one condition field that arrived as undefined, and the request is syntactically perfect and semantically "give me everything".

So the two failure modes are a matched pair, and they push in opposite directions. A malformed multi-value facet gives you five per cent of what you asked for. A missing narrowing filter gives you six hundred thousand studies you did not ask for. Both arrive as HTTP 200 with a well-formed body, and if you are paging through results and paying per row or per compute-second, only one of them shows up on an invoice.

The defence is unglamorous: refuse to send a search that has no narrowing filter at all, and record the query you actually sent on every row you emit. I keep the resolved filter string in a column on every output row, which means "why does this dataset only have phase 4 trials" is answerable by looking at the data instead of by re-deriving what the code would have built.

Zero and "I did not understand you" arrive in the same shape

One more behaviour in the same family, because it is the reason a phase bug can survive a spot check. A condition the registry has never heard of does not produce an error:

?query.cond=zzznotadisease&countTotal=true
-> HTTP 200, totalCount 0

Measured 2026-09-10. A typo in a condition name is therefore indistinguishable from a disease nobody is studying, and both are indistinguishable from a facet value that got discarded down to nothing. Three different problems, one response shape.

This is why I stopped treating an empty result as a result. In my client, "the search ran and matched nothing" is one outcome, "the registry rejected a filter value and here is its reason" is a second, and "the request failed" is a third; they are separate labelled rows, and none of the three is billed. That distinction costs about fifteen lines, and it is the difference between a user filing a bug and a user assuming the data is correct.

A facet language hiding inside a query string

Strip the specifics away and this is a general shape worth recognising. aggFilters is not really a parameter; it is a small expression language with two operators, packed into one URL parameter, with no grammar echoed back in the response and no validation on the way in. The API cannot tell the difference between a list you meant as OR and a list you wrote out of habit, so it applies a rule you have never read.

I have hit the same shape from unrelated sources often enough to have a checklist now. It is three questions, and I ask them of any filter parameter that accepts more than one value:

• Does the count go up when I add a value? Run each value alone, then together. A union must be at least as large as its largest member. This one assertion catches the comma bug in a single run.
• Does the parameter exist? Send it once with a deliberately invalid value. If a bad value is accepted as quietly as a good one, the parameter is probably being ignored entirely — and if it 400s, you have just confirmed it is real.
• Can this query return everything? Delete the filters and look at the count. If the answer is the whole database, then no code path may be allowed to send an unnarrowed search.

Two of the three take one request each. Both of the bugs in this article would have been caught before any code was written.

The request to send instead

Concretely, for "cancer trials in phase 2, 3 or 4", the request that returns 51,889 studies rather than 2,598 is:

GET https://clinicaltrials.gov/api/v2/studies
  ?query.cond=cancer
  &aggFilters=phase%3A2%203%204
  &countTotal=true
  &pageSize=1000

Values of one facet joined with a space; facets joined with a comma; no filter.phase; at least one narrowing filter present; and countTotal=true so you can compare what was declared against what you received. Worth knowing while you are in there: pageSize is capped at 1,000, and asking for 2,000 returns 1,000 with no warning, which is the third member of the same silent-clamp family.

These rules are wired into a public tool if you would rather not write them again — ClinicalTrials.gov Scraper & API builds the facet string through that grouping function, refuses to run an unnarrowed search, pages against what actually came back rather than what was requested, and flattens the twelve nested modules of each study into one row. It also passes start dates through exactly as the registry wrote them, with a second column saying whether the value is precise to the day or only to the month — of 200 cancer trials measured on 2026-09-10, 117 had a full date, 81 had only a year and a month, and 2 had none, so parsing them into a date type invents a day that was never published.

Published by GRAMSHIFT. The counts in this article come from requests sent to clinicaltrials.gov/api/v2 on 2026-09-13, plus two figures explicitly dated 2026-09-10; each one is shown next to the request that produced it, so you can re-run it. Disclosure of AI use: the probe scripts and the first draft of this text were written with Claude Code, and every number here was then read back off the raw HTTP responses before publication rather than taken from the draft.