Why the data matters so much
An AI model does not start with knowledge and then get a few corrections. What it can do is mostly a product of the examples it was trained on. A system that learned from millions of captioned photos can describe photos. One that learned from large amounts of written text can write in many styles. If a topic, language or kind of person is rare in the examples, the model usually handles it less well.
So when people ask where an AI “gets its information”, the more accurate answer is that it does not look things up in a library of facts. It has adjusted its internal numbers to reflect patterns in training data. The mechanism is described in the explanation of how language models work. This article is about the material that feeds that process.
Common sources
Companies seldom publish full details of what they use, so descriptions here are general and the mix differs between models.
Text from the public web. Large collections of web pages, articles, forum discussions, blogs and reviews are a main source for text models. Some are gathered by automated programs that visit pages and copy their content.
Books and long documents. Books, academic papers and reports give models examples of extended, structured writing. The status of some book collections has been disputed, as discussed below.
Reference material. Encyclopaedias, manuals, question-and-answer sites and public records help with factual and instructional content.
Code. Public code repositories and documentation teach models about programming languages and how people write and explain software.
Images with captions or alt text. Image generators learn from large numbers of pictures paired with descriptions, often collected from the web. How these are used is covered in the article on image generators.
Audio, video and transcripts. Speech and video tools use recordings and their transcriptions.
Licensed and purchased data. Some companies pay to use archives from publishers, image libraries or other owners.
Data created for training. Companies commission people to write example questions and ideal answers, or to rate which of two replies is better. This human feedback is used to shape how a chat tool behaves, such as making it more helpful and less harmful.
Synthetic data. Output from one model is sometimes used to train another. It can fill gaps but may also pass on and amplify the first model’s errors.
User interactions. Some services may use conversations to improve their products, depending on terms and settings. See the article on chatbot privacy for how that works.
How the material is prepared
Raw collections are messy, so they are processed before training. Typical steps include:
- Removing duplicates, since repeated text can make a model overweight it.
- Filtering for quality, discarding gibberish, spam, broken pages and, as far as possible, harmful content.
- Removing some personal information, although this is imperfect.
- Splitting into pieces the model can work with, such as tokens for text.
- Choosing a mix, deciding how much of each kind of source to include.
Every one of these choices is made by people and shapes the finished model. A filter that removes “low-quality” pages might also remove dialects or writing styles that some communities use.
What the data does not guarantee
Large does not mean reliable. The web contains errors, outdated information, opinions presented as fact and deliberate falsehoods. A model absorbs patterns from all of it, which is one contributor to the behaviour in the article on AI hallucinations.
Training also has a cut-off. A model’s built-in knowledge stops at some point, and it may not know about later events unless it is connected to a search tool or given fresh material.
Coverage is uneven. English and widely discussed subjects are typically better represented than small languages and specialist topics, so quality can differ by language and subject.
Bias that comes with the sources
Writing and images online reflect who has had the means and the inclination to publish, and they carry the stereotypes and imbalances of the societies that produced them. A model trained on such material can reproduce those patterns. Everyday examples are collected in the article on algorithmic bias.
The open questions
Several debates around training data are unresolved, and legal positions differ by country and are still being tested in courts. It is not possible to say here what is lawful in any given place.
Consent and credit. Many writers, artists and photographers did not specifically agree to have their work used. Some object, especially where a model can imitate a living creator’s style.
Copyright. Companies and rights holders disagree about whether training on copyrighted material is permitted under existing law or requires a licence. Lawsuits and new rules are ongoing.
Privacy. Public web pages can contain personal details. Whether collecting them for training is appropriate, and how people can ask for removal, are live issues. The broader problem of how companies collect information about people is discussed in the article on data brokers.
Opt-outs. Some websites use technical signals or settings to ask crawlers not to collect their content, and some companies offer opt-out forms. Whether these are honoured, and whether they work after the fact, varies.
Transparency. Without clear lists of sources, outsiders find it hard to assess quality, bias or legal risk.
Models training on model output. As more web content is machine-generated, there is concern that later models will learn from earlier models’ mistakes. How serious this will be is still being studied.
What this means for a user
You do not need to know exactly what went into a model to use it sensibly. It helps to remember three points. The tool reflects its sources, flaws included. It may be weak on topics or languages that were scarce in training. And your own inputs may or may not be added to future training, depending on the service. For anything where accuracy or fairness matters, verify the answer elsewhere and read the provider’s stated policies.