AI Source Scraper Autor: Janna Joceli Omena

AI Source Scraper
A research tool for capturing and comparing sources surfaced by AI search interfaces.

Designed for digital methods and AI search research, the extension turns source and activity panels from supported AI platforms into structured datasets while preserving information about the conditions under which each response was generated.

Rather than collecting links alone, AI Source Scraper lets researchers annotate each capture with methodological metadata:
- Capture mode: Longitudinal or Comparative
- Search condition: AI search or Agentic AI search
- Prompt condition: Isolated, Initial, or Follow-up
- Prompt framing: Underspecified, Program, Anti-program, or Ambiguous
- AI fieldwork observations: researcher notes recorded during data collection

Captures accumulate within a research session and can be exported as CSV, JSON, and TXT, allowing researchers to merge, filter, compare, and analyse source-selection patterns across prompts, platforms, search conditions, and moments in time.

This research tool operationalises digital methods such as the [Longitudinal Prompt Method for Understanding AI Search] (https://www.researchgate.net/publication/408580170_Longitudinal_Prompt_Method_for_Understanding_AI_Search), supporting projects that seek to advance source studies of single or cross-AI models by turning the citation layer of AI chatbots into structured data.

Two research designs

Longitudinal captures are designed to investigate how AI responses, sources, or interfaces develop and change over time or across conversational sequences.

Comparative captures support research designs in which conditions are deliberately varied, for example by comparing AI search with Agentic AI search or testing different prompt framings.

The same metadata structure is available in both modes, making it possible to build comparable datasets while keeping the methodological conditions of each capture explicit.

Supported research contexts
AI Source Scraper is intended for research involving AI-mediated web search, source selection, citation practices, conversational search, and the changing relationship between prompts, search conditions, and retrieved web sources.

ChatGPT and Gemini can be studied in logged-in or logged-out conditions where their interfaces permit anonymous use. Claude.ai requires an account to conduct conversations.

Data and privacy
The extension processes captured research data locally in the browser. Researchers decide when to export or clear their session data.

Citation
Omena, J. J. (2026). AI Source Scraper [Computer software]. Zenodo.
https://doi.org/10.5281/zenodo.20945556

Source code:
https://github.com/jannajoceli/ai-source-scraper