Define the question families before choosing a source
Extractive questions ask for a value or label present in a source. Comparative questions ask which category is larger or whether a change occurred. Arithmetic questions require operations such as difference, ratio or percentage growth. Structural questions concern the relationship between headers, rows, legends and footnotes. Explanation questions can demand evidence that a chart does not contain. Label these families separately so a high score on easy lookups does not mask failures on computation or unsupported explanations.
The same source can support several families, but do not multiply nearly identical questions and call that independent coverage. Track source, template, operation and entity groups. Buyers should know whether a collection contains varied business situations or repeated paraphrases over a small number of charts. Decide whether the source is an image, a parsed table, both, or a larger document context.
Read research comparisons as task definitions
ChartQA official repository distinguishes human-authored and machine-generated question files. FinQA official repository supplies reasoning-program and supporting-fact fields. TAT-QA official repository combines tabular and textual financial context. Each reference illustrates a different supervision choice; none makes all chart, table and business reasoning tasks equivalent.
When comparing these resources with a new collection, examine the input available to the model and the expected output. A task that provides a clean table does not fully test chart reading from pixels. A finance-focused example does not establish coverage of operational dashboards or retail category reports. Check release-specific terms and source-document conditions independently from the repository’s software license.
Preserve the information a human reader uses
A useful chart record can need title, axes, tick labels, legend, series, colors, scales, units, data labels and notes. A cropped plot may exclude the footnote that changes the meaning of a number. A parsed table can lose merged headers, indentation, signs or row-group boundaries. Preserve enough of the original source and its context to review the answer without reconstructing missing semantics from guesswork.
Indicate whether underlying numeric data is available. If a value was estimated from a raster chart rather than taken from a source table, represent that process and its tolerance. Do not label an approximate visual estimate as an exact source value. For tables, preserve empty, suppressed, not-applicable and zero cells as distinct states when the source distinguishes them.
Units, denominators and rounding belong in the reference
An answer such as 25 is incomplete when the question asks for a percentage change, an amount in thousands or a ratio. Store the answer value, unit, source scale and permitted formatting. State the denominator used for percentage calculations and whether the target is percentage points or relative percent. A correct arithmetic expression can still answer the wrong business question.
Write rounding rules before adjudication. An exact extraction from a published rounded table may not equal a figure recalculated from more precise underlying data. Explain which source level is authoritative for the task. For multi-currency or multi-period tables, avoid comparisons that require an unstated exchange rate or inflation adjustment. If those conversions are intended, supply the reference rate, date and transformation explicitly.
A worked example that separates arithmetic from causality
The invented download contains revenue of 120 and 150 in USD thousands for two periods. Relative growth is the difference, 30, divided by the original 120, multiplied by 100: 25 percent. A change of 30 thousand dollars and growth of 25 percent are both valid descriptions of different targets. The question wording and answer schema determine which is expected.
A second invented question asks what caused cost to rise. The table shows a change but no causal explanation. The correct reference is unsupported, not a fluent guess about staffing or materials. Including this case tests whether an answer system can distinguish a visible relationship from information absent from the source. Neither example is a result from testing a model or a real company disclosure.
Evidence annotations should be auditable
For tables, cite row, column and cell identifiers, together with the source revision. For charts, an evidence region may need links to a series, label and source table. For hybrid documents, identify the paragraph or footnote supporting an interpretation. Keep evidence references separate from free-text rationales so an evaluator can check them programmatically and a reviewer can inspect their visual meaning.
If several derivations are acceptable, describe equivalence rather than forcing one string form. A program representation can make arithmetic reviewable, but validating syntax does not prove that the selected inputs answer the question. Review the evidence selection, operation and final expression as distinct layers. Incorrect intermediate evidence should not disappear merely because a coincidental final number matches the reference.
Design difficult cases deliberately
Useful coverage can include repeated labels, inverted axes, truncated scales, stacked totals, negative values, sparse tables, merged headers and footnote-dependent definitions. These should represent actual task conditions, not trick questions introduced only to make a collection seem challenging. Keep a coverage matrix identifying which conditions occur and how many independent sources support each slice.
Include unanswerable or ambiguous examples under a written policy. A question can name a category absent from the chart, ask for a period not shown, require unavailable precision or refer to an unreadable label. Store why the answer is unavailable. Distinguish an intentionally unanswerable case from a damaged or incomplete source that should have failed delivery acceptance.
Split sources and review agreement
Group charts or tables derived from the same report, template or underlying numeric series before splitting. Alternate image renderings and textual transcriptions of one table can leak across evaluation if treated independently. Consider holding out report families, organizations, chart types or time periods according to the model’s intended use. Report those choices rather than relying on a random-row split.
Use separate review for answer correctness and evidence sufficiency. Inspect disagreements around units, interpretation, rounding and missing values; these often reveal an incomplete rubric. Agree a correction process and retain examples changed during adjudication. A collection should expose known limitations, including source OCR or chart estimation errors, rather than presenting all targets as equally certain.
Quote useful supervision rather than question count alone
Ask whether pricing covers source access, chart/table extraction, question writing, evidence annotation, derivation programs, visual review and corrections. A lookup over a clean table can require much less work than a footnote-dependent question over a scanned report. Existing benchmark access, custom annotation labor and a commercial source license are separate price components.
For a pilot, identify the task mixture and accepted question-source pairs, then estimate review workload from representative examples. Do not turn a chosen labor rate into a claimed market price for chart data. A Fiund proposal depends on the actual source, intended use, required evidence, annotation complexity and delivery conditions. The downloadable review sheet helps make these assumptions explicit before comparing offers.
Do not confuse plotted percentages with percentage change
A bar increasing from twenty percent to thirty percent changes by ten percentage points, while its relative increase is fifty percent. A reference should state which operation the question requests. Keep the source’s denominator and population definition with the annotation, particularly when comparing different reporting periods. The correct arithmetic cannot repair a comparison between incompatible populations or definitions.
Download the worked specification
The worked question and review sheet use invented chart/table values to demonstrate evidence, units and calculation checks. Their numeric answers are consequences of the illustrative inputs, not financial facts or benchmark results. These files contain no licensed chart collection. Preserve the distinction between source cells, derived calculations and reviewer decisions when adapting the example to an actual QA pilot.
Questions buyers ask
Sources & research notes
Public documentation reviewed 2026-11-13. Provider descriptions and published terms are attributed below. Published facts are attributed to their sources. Worked files are labeled as illustrative or as an inspected editorial source; neither implies commercial dataset clearance.
- ChartQA official repositoryChartQA authors · Reviewed 2026-11-13
Primary publisher documentation reviewed for the linked claim. Benchmark access, code terms and commercial source permissions are separate; no benchmark is represented as Fiund inventory.
- FinQA official repositoryFinQA authors · Reviewed 2026-11-13
Primary publisher documentation reviewed for the linked claim. Benchmark access, code terms and commercial source permissions are separate; no benchmark is represented as Fiund inventory.
- TAT-QA official repositoryTAT-QA authors · Reviewed 2026-11-13
Primary publisher documentation reviewed for the linked claim. Benchmark access, code terms and commercial source permissions are separate; no benchmark is represented as Fiund inventory.