Creating AI flashcards: what works and what doesn't

You have a thirty-page lecture handout, the exam is in three weeks, and somewhere in your head a voice whispers: "Just let the AI do it." A good idea, halfway. AI can take the typing off your hands. It cannot take off the job of deciding what a good card is, and sometimes it invents answers that sound convincing and are wrong.

This article sorts out what research actually knows about AI-generated study questions (less than the marketing claims), what the difference is between a model on your iPhone and one in the cloud, and how to check AI cards so you don't spend weeks repeating nonsense.

What the studies say about AI questions

Short version: quality varies widely, and there is no credible figure like "AI cards are X percent wrong". The studies each measure something different.

The sobering early result. Ngo et al. had ChatGPT 3.5 write 60 multiple-choice questions from supplied teaching texts and checked them for content. Only 19 of 60, roughly 32%, met all requirements for a correct question, answer, and explanation. In 25%, the answer was wrong or misleading. Important: the remaining flaws were partly missing explanations. "68% hallucinations" would therefore be an incorrect summary, even if you find it online.

The direct comparison. Laupichler et al. gave 161 students neurophysiology questions in random order, 25 written by humans and 21 by ChatGPT 3.5. Difficulty did not differ significantly. But the human questions discriminated better between students who had mastered the material and those who had not (discrimination index 0.36 versus 0.24). Translated: the AI questions were not harder, but they were worse at checking whether someone had really understood the material.

The more positive result with a newer model. Safadi et al. had 31 experts blindly rate 15 questions each from o1-preview and 15 from experts on mechanical ventilation. The AI questions were not worse than the predefined threshold. Sounds good, but it is a small dataset rated by experts, not by learners. It says nothing about your learning success.

The only direct flashcard study. Closest to your everyday use is "Memory Machines" by Kirkby and Matuschak (2026). They had language models generate flashcards from highlighted text: 1,499 card variants from 475 highlights and 93 sources. 37.8% were considered directly suitable for repetition. The authors themselves say there is no long-term validation, and the ratings were mostly done by themselves and a small circle of experienced users. Whether the report was peer reviewed could not be verified. The figure is therefore not a success rate of today's AI, but a signal: even under good conditions, a large share of the cards needed rework.

What you lose by outsourcing: the generation effect

There is a second problem, independent of whether the AI is right. In five laboratory experiments with 96 students in total, Slamecka and Graf showed in 1978 that words you produce yourself stick better than words you only read. The meta-analysis by Bertsch et al. (2007) across 86 studies and 445 effect sizes found an average advantage of 0.40 standard deviations. That does not mean "40% more learned". And the effect depends on design: 0.50 when the same people went through both conditions, 0.28 for comparisons between groups.

Michael Nielsen writes in "Augmenting Long-term Memory" that formulating good questions is itself part of understanding, and recommends your own decks. Andy Matuschak describes writing cards as interpretive work: selecting ideas, recognising connections, adapting questions to your own prior knowledge.

To be honest: these studies examine words, not flashcards. That AI cards are worse in the long run does not follow from this . You can spend the saved time on repetition. What is plausible is only this: those who no longer formulate anything themselves probably forgo something. And Matuschak does not reject automatically generated cards; he himself investigated them in "Memory Machines".

On-device or cloud?

For AI flashcards there are roughly two routes: a model on your device or one on a server.

Apple's on-device model

Apple introduced the Foundation Models framework at WWDC 2025. Since iOS 26 and iPadOS 26, apps can use the local language model. What you should know:

• Privacy: According to Apple, inputs and outputs of the local model stay on the device. It works offline and has no usage-based fees.
• Devices: iOS 26 alone is not enough. You need a compatible device and Apple Intelligence enabled. According to Apple, this includes iPhone 15 Pro and Pro Max, the iPhone 16 models and later compatible devices, iPads with M1 or newer, and iPad mini with A17 Pro. Language and region settings must also be supported.
• Size: The local model introduced in 2025 has roughly 3 billion parameters. Apple describes it as optimised for summarising, extracting, and classifying, explicitly not for broad world knowledge or complex reasoning. Since June 2026 Apple speaks of a whole model family; the statement applies to the original framework model.
• Context: Per session, 4,096 tokens fit, input and output together, including instructions. A long lecture handout must therefore be split into sections.

The description "good at extracting, poor at world knowledge" is actually good news for flashcards. Let the model make cards from your text, not from its memory. And structured output only means that question and answer arrive cleanly separated. It does not guarantee that the answer is correct.

A cloud model with your own key

Large cloud models know more and handle longer texts. The cost is that your text goes to a server. With OpenAI you pay per token, separately for input and output; the prices are listed per one million tokens. There is no flat price "per card". An example from the price list (as of November 2026): for gpt-6.1-terra, 10,000 input tokens plus 2,000 output tokens together cost 0.02 US dollars, without further services.

On privacy: according to OpenAI, API data has not been used for training by default since 1 March 2023, unless you explicitly opt in. No training does not mean no storage, however. Abuse-monitoring logs may contain prompts and responses and are kept by default for up to 30 days.

How checkst does it

checkst prioritises the local model first. If your iPhone or iPad supports Apple Intelligence, checkst generates the cards on the device. If you prefer a cloud model, you store your own OpenAI API key. It stays on your device in the keychain, and billing happens directly between you and OpenAI. If you have neither Apple Intelligence nor a key, checkst copies a ready-made prompt for you: you paste it into an AI app of your choice, copy the answer back, and checkst turns it into card suggestions.

The process:

1. Choose source: paste text, open a PDF, or select images, such as photos of book pages or board notes. For images, checkst reads the text via text recognition.
2. Choose card type: basic (question and answer), cloze deletion, or mixed. The cards are created in the language of your source.
3. Generate: checkst automatically splits long texts into sections, smaller for the local model than for the cloud, precisely because of the context limit mentioned above.
4. Review: You see every suggested card and decide individually which stays. Only what you select is saved.

Afterwards the cards land in a deck and are scheduled with FSRS like all the others.

How to check AI cards in two minutes

Step four is the most important. These questions help while going through them:

• Is it stated that way in your source? If you cannot find an answer in the lecture handout, out it goes. This is the most common error.
• One thing per card? AI loves cards with three answers at once. Split them or delete them.
• Does the question give away the answer? Then you are only practising reading.
• Does the card test whether you understood? Remember the discrimination finding from Laupichler et al.: AI questions are often easy enough but too superficial.
• Would you still understand it in three months? Without context from the lecture handout, "What is the second feature?" becomes unanswerable.

And rewrite a few cards yourself instead of just nodding them through. It costs little and gives you back part of the generation effect.

The first card is typical of an unchecked AI draft: a vague question, an answer as a keyword salad, nothing of it clearly testable. The two below each ask exactly one thing and can be checked against the source.

Conclusion

AI is a good intern for flashcards: fast, diligent, sometimes convincingly wrong. Let it draft from your material, ideally on the device if yours supports it. Check every card against the source, throw out generously, and reformulate a few yourself. Then you have the time savings without writing errors into your long-term memory.