CASE STUDY
Catholic Guardian Services
Seeing a brand the way AI sees it
BRAND
Catholic Guardian Services (CGS) · 100+ years old, $100 million in revenue (FY2025), publicly and privately funded, comprehensive human services organization in NYC.
CATEGORIES TESTED
Foster care & adoption · Children’s mental health · Family support & crisis response · Developmental disabilities · Faith-based giving
1. The paradox
CGS relies on the accuracy and visibility of its brand to recruit foster parents, raise money, and inform the agencies it works with. On the eve of a capital raise in 2026, CGS saw that, when asked about CGS by name, AI called it “an excellent and highly reputable charity to donate to.” But when asked for the top providers in NYC of the 5 services CGS offers, without naming any agency, CGS was rarely named in AI responses.
“Becoming easier for AI to find and describe us cannot come at the cost of who we are. This work is allowing us to be discovered while preserving our unique identity and value to humanity.”
2. The System
AI answers questions using different sources of information. The System asks the questions that a potential customer or donor would ask. Several prompts name the brand directly; most only ask about the category. My focus is the sources behind the AI responses. While brands cannot control AI, they can influence some of its sources, if they know what they are, how often AI uses them, which questions produced them, and the language AI used in the answer.
That is why I record every answer and every source it cites. The System opens and reads those sources to check if they are real and accurate. Some are neither, which matters quite a lot to businesses wanting to improve how AI represents them. The sources must be read, measured, and mapped for brands to see themselves and their competition in AI systems.
3. The investigation
In June 2026, a CGS working group of the Chief Development Officer and Chief Strategy Officer, reporting to the CEO, agreed on the questions to test: 6 that a potential donor might ask, without naming CGS; 4 that asked about CGS directly. 3 AI systems, plus Google’s AI Overviews, answered each question 3 times. In October, the questions rose to 20, and the repetition of each question increased to 8 per system.
4. What I found
In the 6 original category questions, CGS was rarely named, except in answers to the faith-based question.
01
CGS was listed in some trusted and authoritative third-party sources, but not in language that could tell AI which providers are “the best.” Those sources list; they do not recommend.
02
Many government registries and partner agencies listed CGS by its former name and address, or didn’t list it at all.
03
CGS has 3 websites focused on services, but none presents a comprehensive list to AI. Several of those services were described under branded program names and without a consistently stated NYC focus.
04
The donor question. Asked where a donation would do the most good for children in foster care in New York, AI named CGS in none of 27 answers. 16 of the 27 cited Charity Navigator profiles, but only those of other organizations. CGS has a profile there too: when asked about CGS by name, AI cites that profile in more than half of its answers. When asked where to give, it never did.
05
A public record AI wasn’t using. In June, a separate search of one NY State agency’s website found 19 documents naming CGS. The AI answers cited only one of them. In October they still cited only that one: the agency’s directory of foster care agencies, which still listed CGS at its old address.
06
Citations that led nowhere. Of the 1,201 web addresses AI cited that I tried to open, 146 returned “page not found.” For example, AI cited 13 different CauseIQ pages for CGS, each under a different, wrong tax ID. None of them exists.
07
A source growing on its own. An automated update by a Wikidata editor unknown to CGS added 13 years of Form 990 financial data to the CGS entry, more than doubling its size. The record is being built by others as well.
08
5. What CGS changed
New service pages were designed, listing and describing every service in one place, using terms funders and families use, and naming NYC as the focus.
The agencies that fund and license each program were added to each service description.
All 3 websites received structured data identifying them as one organization.
The CGS Wikidata entry was updated, and updates to Wikipedia have been submitted.
A newsroom launched, with each announcement anchored by the same statement that names the organizations that fund and license CGS’s major services.
Partner sites that still list CGS with outdated information are being updated.
What changed by October:
AI names Wikidata more often. AI does not cite Wikidata directly. But when asked after the category prompts which sources informed its answer, it named Wikidata far more often in October than in June.
A correction that moved AI somewhere else. Catholic Charities of New York, an umbrella organization of over 90 agencies including CGS, was cited in about half of the June answers to questions asking about CGS by name. At CGS’s request, Catholic Charities updated its CGS profile page. In October, AI all but stopped citing that page, and instead began citing much more often an outdated article about CGS on the same site. Only reading the sources showed this.
A new page AI found within weeks. A post on the new newsroom announcing CGS’s accreditation renewal did not exist in June. AI began citing it in October.
* * *
Knowledge transfer. The most valuable change is not yet measured in numbers. By learning the process of measuring and improving its brand in AI systems, CGS can be confident it will be in a strong position to continue serving the people of NYC for the next 100 years.
6. How I know
The protocol. The client approves the questions before they are asked. Every answer and every source it cites is kept. Every count I publish is read by two AI readers working to the same written rules; I settle every disagreement and read a sample of the answers they agree on. No claim that a brand is absent is published without a reading. When a rule changes, earlier quarters are recounted before they are compared.
What the System read in October. AI cited 1,905 distinct web addresses. 704 were pages owned by organizations in the category. I keep track of them, but set them aside when plotting where a brand stands against its competition. I tried to open the other 1,201. 187 would not open, 146 of them “page not found.” Of the pages that opened, 94 defined the category. “Category-defining sources” name three or more organizations from the CGS-approved competitor list, or list five or more providers in the region.
The System is designed to scale up or down with the needs of the client. The rules are written down, each quarter is run by them, and any change is dated and disclosed.
Correcting the June baseline. The first June count included answers to an internal follow-up question asked by The System after each category question, which inflated the total. Each answer was then read, and I corrected the June figures before any comparison was made. In the revised count, CGS was named in 12 of June’s 72 category answers and in 2 of the 12 answers specific to foster care. Those are the June figures used throughout this case study.
Rigor. In August 2026 the Interactive Advertising Bureau published “Measuring Visibility in the AI Era,” guidelines for this kind of measurement. It states that a program with fewer than 50 questions is exploratory. So for the October audit, I expanded the foster care questions from one to eleven, and asked each question eight times instead of three, except on Google’s AI Overviews, which are captured by hand and so were still asked three times. Each of the 16 category questions was answered 27 times: 432 answers, of which the original six questions account for 162. Every answer was read twice.
The extra questions mattered. One June question had found CGS in 2 of 12 foster care answers. Ten more specific questions, asked in October, found it in about half of the answers. They are different questions, so this is not a trend. The answers to a single question can under-represent a brand. This is a dramatic example of why the IAB calls for a large and varied set of questions.
The limits, stated plainly. The IAB framework asks measurement providers to disclose how their numbers are made and where they fall short. I abide by that. These are limits on what the findings can show, not on whether the counts are right:
The answers come from the AI systems’ programming interfaces (APIs), plus Google’s AI Overviews captured by hand from a New York connection. People using chatbots may see different answers.
The October run asked 20 questions. Under the IAB framework, a program needs 50 to count as more than exploratory. Q4’s measurement for CGS in January should meet that threshold.
Some websites block automated readers or only show them a few characters. Any page captured at fewer than 1,500 characters is re-read in a browser that waits for the full page to load. Pages that still don’t open are checked by hand or disclosed as not read.
Two AI readers read all 540 answers to the CGS prompts in October - 432 to category questions and 108 to prompts asking about CGS by name. I settled every disagreement and read a sample of the answers they agreed on. One of them, ChatGPT, is also a system being measured; its reading task is mechanical, and this is disclosed.
7. What happens next
The October measurement asked the original six questions in the same words as June. CGS’s share was essentially unchanged - as expected this early. Its information on partner websites, as well as its own, was still being fixed during the October measurement. This case study documents the start of a correction for CGS, not a result.
The new CGS service pages launch in late October. My prediction for Q4: once the consolidated CGS service pages launch, changes to Wikipedia are accepted and implemented, and Catholic Charities updates its news article about CGS, AI will associate CGS more strongly with its services, and name it in more of the answers to the original six questions than October’s 29 of 162. If those three changes are not in place by January, I will say so, and the prediction will carry over to the next quarter. The measurement and the strategy behind the changes are pro bono.
MICHAEL QUINN, OCTOBER 2026.