Data Mint

FAQ

The questions people ask when they start, grouped by where they come up in the work.

Signing in

How do I sign in?
Enter your email and click Send sign-in code. The email holds a six-digit code. Type it into the browser you want to work in. The code works for two hours and only once. If you would rather click a link, choose Prefer a link? Email me one.
Do I have to sign in again on a different computer?
Yes. Everything you made is on the server, so you land where you left off. Do not sign in on a computer that is not yours, and sign out when you are done.
My work computer blocks the site.
Use a personal computer. Some corporate networks block the domain and the mail both, and nothing on our side gets through that.
The sign-in email never arrived.
Some university mail servers drop our email without telling you. Sign in with a personal address.
If my computer crashes in the middle of a mint, do I lose anything?
No. Minting happens on the server. The only thing you can lose is text you were typing into a box and had not submitted.
Who owns the documents I upload?
You do. You can delete any document at any time. The Data Policy page names every AI provider that can read a document and says what each one keeps and for how long.

Documents

Does the site find documents for me, from PubMed or anywhere else?
No. You bring the documents. We looked at building connectors, and the space of sources is too big.
Why did one file turn into 216 documents?
CSV, Excel, and RIS files become one document per row, because that is usually what you want. Every other file is one document. Before importing, the site shows the first row and asks whether it holds column names.
Can I upload a zip file?
Yes. Each file inside becomes one document.
Can I mix PDFs, text files, and images in one project?
Yes. Mint documents with selectable text as Text, and scans and images as Vision. A scanned PDF is a picture of a page. It has no text to copy.
How do I tell whether a PDF is scanned?
The site marks it Scanned in the document list, and offers Vision when you mint it as text. The mark is sometimes too eager. Click the document and read the extracted text. If the text is there, mint it as text.
I have the same filing as a formatted PDF and as plain text. Does the format matter?
Not much. Text extraction from a PDF that was made on a computer is very good. Use Vision only if you need what is in the figures. EDGAR filings saved as .txt work fine.
Are twenty research articles twenty documents or one?
Twenty documents, unless all twenty report one trial and the trial is what you are measuring. Fifteen interviews are fifteen documents and fifteen rows.
Does a reader ever see two documents at once?
Never. Every document is read on its own. If you need information combined across files, join the files into one document before you upload it.
If I add documents later, does that disturb the ones already minted?
No. A project holds documents and codebooks. New documents show up unminted until you mint them.

The codebook

Does the codebook get made for me?
You make it with the assistant. You describe the project. It asks clarifying questions and proposes the fields. Skip the questions and you still get a codebook. Answer them and you get a better one.
Can I give it a paper that defines my construct, or an example document?
Give it to the assistant with Attach files, and it drafts from it. The readers see only the codebook, so anything they need has to end up in the codebook text. To teach by example, paste the example into the item with the reasoning, not only the answer. Two examples can pull every reader toward those two cases, so say that other cases will look different.
Can I add a question after the codebook is made?
Yes. The easiest way is to go back to the assistant and describe what you want added. Editing by hand with Direct Edit works too.
Can it give me a 1 to 5 scale, or numbers I can run a regression on?
Yes. Say the format in the codebook item. Also say what to write when the document does not answer the question, such as NA.
What do "Take me literally" and "Read between the lines" do?
They set how far the assistant goes beyond your own words. "Take me literally" writes what you described and nothing more. "Read between the lines" is the default. Where it infers something you did not say, it puts that reading in a Hint, so you can check it. Start there. "Thorough" is greyed out while it is tested.
My codebook has seventy notes now and no person could follow it.
That is what overfitting looks like. Paste the whole codebook and the original paper back into the assistant, and ask it to turn the notes into a tighter set of items. Do not add a seventy-first note.
Is revising the codebook until it works a kind of p-hacking?
It is fine as long as you are not looking at all of your documents while you revise. Hold some back.
Can I put a picture in a codebook item?
No. Codebooks are text only. That is a real limitation.
Why do answers get worse when I ask for more variables per document?
A reader can only write so much in one answer. So the assistant splits a large codebook into phases. A reader answers one set of items, sees its own earlier answers, then goes on to the next set.
Is a codebook the same thing as training materials for human coders?
Yes. Not sharing them is bad science. "We measured something, and I won't tell you how" is absurd.
Can anything write the codebook for me?
Nothing can, and nothing should. Writing the codebook is the measurement.

Minting

Are Readers A, B, and C different roles?
No. All three get the same codebook and the same instructions. They are different models, from different labs. In Pro they are three Gemini models of different sizes.
Do the readers see each other's answers?
No. Each reader gets the codebook and one document, alone.
What reads the three answers afterwards?
An arbiter. It reads the three answers, finds where they disagree, and proposes what in the codebook would settle it. It does not pick a winner. We tried that early, and it did a bad job.
Won't more readers mean more disagreement?
Yes. That is the point. It does slow the codebook work down, because you get more disagreements to sort through, and some of them are trivial.
Should I click Mint All?
Not while you are still working on the codebook. Click mint on five to ten rows, or use Test First 3 Documents when it shows. Read the results, revise, and mint again. Mint the rest when agreement is high. You will run through your mints fast otherwise.
Why does a mint take twenty minutes?
Usually because it is waiting in line behind other people's jobs. The three readers run at the same time, and your documents run in parallel with each other. Speed depends on how busy the models are that day. What to expect has the numbers.
When should I use Pro?
For very long documents, and for codebooks with twenty or more items. Pro costs five times as much as a standard consensus mint.
What does Squint do?
It turns a scanned list, such as a ledger or a registry page, into one row per entry. It reads one page per document, and it is open by invitation for now. Try Vision first. Squint sends zoomed images of each page and gets expensive fast. Write to karl@datamint.ing if Vision is not enough.
What does scientific minting do with equations and tables?
It prepares the document before the readers see it. We call that blooming. Equations become LaTeX, which the models read the way they read Spanish. Tables and figures become text.
I have minted, and now I want to change the codebook. Do I start over?
No. Save the change as a new codebook version. The old version keeps its results. The spreadsheet shows one codebook at a time. Switching codebooks changes the columns and deletes nothing.
After I change the codebook, do I have to re-mint everything?
That is your choice. A new version is a new codebook, and the readers have no memory of the old one, so there is no partial update. Re-mint the documents you want answers for.
Can I stop a mint?
Open the ⋯ menu on the row. If Cancel is available, the job has not started yet. Cancel it, and all its mints come back. If Cancel is greyed out, the readers are already at work and the job cannot be stopped. When it finishes, choose Clear results or Request refund from the same menu.
What do I do with a failed job?
A failed job is refunded in full. The row says why it failed. Open the ⋯ menu on the row, choose Clear Minting, and click mint again.

Reading the results

Two readers say "0 items". Is that an error?
No. It means those readers found none of what the question asks for. Every list question should be able to say zero.
Are the certainty percentages a real statistic?
No. It is the reader's own sense of how sure it is. Models are surprisingly good at putting a number on that, but it is not a calibrated probability.
When readers disagree, how do I know whether it is my codebook or a weak model?
Read the quote and the reasoning each reader gave, and decide. Nothing stops you from rejecting how a reader answered. If a cheaper model cannot follow your codebook, that tells you something, because most people who use your codebook later will not have the most expensive model either.
Which disagreements should I bother to resolve?
For each one, ask how much you care. Fix one and another comes up somewhere else. That judgment is the work. It is why you do it with colleagues, and why you publish the codebook.
Is there an objective measure, such as F1, for "good enough"?
We do not believe in a gold standard here. The codebook says what the measurement is. The check against cherry-picking is that the codebook is public, not an accuracy number.
Should I check every answer by hand?
That depends on the stakes. At the least, read the quotes and reasoning for a few documents, and look at whether the three readers agreed.
Will the same codebook give the same answer twice?
No, and we do not force it to. Forcing identical output tends to make the answers worse. Three independent readers are a rough check on how stable an answer is.
How is agreement worked out for a table?
Cell by cell. It gets harder when readers return different numbers of rows. Reading a table is a lot of work for a reader.
Where do I see how a document was prepared for scientific minting?
Click the flower in the document's row on the minting page.

Exporting and sharing

Can I export to Excel?
Yes. On the inspection page, open Export CSV and choose Linked Tables (Excel). Each table in the codebook gets its own sheet.
Does the export show where the readers disagreed?
No. The export carries one answer per cell. The disagreements stay on the inspection page. How to carry disagreement into a dataset is an open problem, and we would like to hear how you think it should look.
Can I load the results straight into R or Python?
Yes. Create a share link. The share dialog lists data URLs you can read from R, Python, or Stata.
What does a share link show?
The inspection page, to anyone who has the link, with no sign-in: every answer, the quotes, the reasoning, and new results as you mint them. It does not open the documents. To work with someone on the documents, choose Invite a collaborator in the same dialog. A collaborator sees the documents and codebooks and can resolve disagreements with you, and cannot spend your mints.
How do I cite this in a paper?
See the Cite page. Put the share link in the paper too, so a reviewer can see the work instead of taking a number on faith.
Do journals require the codebook?
We do not know for every journal. Email the editor and ask. Some labs publish a column header and nothing else.

Cost

How is cost worked out?
One mint is one penny. A consensus mint on text is priced on the pages the readers read and the items they answer. A quick mint on the standard engine costs about a tenth of that. Vision costs more, and Pro costs five times as much. The price shows before you click, and the job cannot charge more than that. What to expect works one example.
Is a second mint of the same document cheaper?
Only for scientific minting, because the document is bloomed once and not again.
What do I get for free?
A new account starts with 2,000 mints. It gets 250 back each day, up to 2,000.
Why not paste the documents into a chatbot?
A chat has no process you can repeat, and you still fill in the spreadsheet by hand. Here you give a codebook and documents and get a spreadsheet back, the same way each time, with the evidence for every cell.

Back to Learn