AI Receipt Scanner

Project Blueprint: AI Receipt Scanner

Subtitle: Digitize and categorize your paper receipts instantly using AI. Difficulty: Beginner Category: Expense Tracking

1. The Business Problem (Why build this?)

In an increasingly digital world, the archaic process of managing physical paper receipts remains a significant friction point for individuals and small businesses alike. The manual handling of these documents leads to a multitude of inefficiencies and potential financial losses.

• Time Consumption & Productivity Loss: Manually sorting, categorizing, and entering receipt data into spreadsheets or accounting software is a tedious, time-consuming task. This burden diverts valuable time from core activities, impacting personal productivity or business growth.
• Data Entry Errors: Human error is inevitable. Incorrectly transcribed amounts, dates, or missed entries can lead to inaccurate financial records, complicating budgeting, tax preparation, and financial analysis.
• Lost Receipts & Missed Deductions: Physical receipts are easily misplaced, damaged, or lost. For businesses and individuals, this directly translates to missed tax deductions, unreimbursed expenses, and incomplete financial audits, leading to tangible financial losses.
• Lack of Real-time Financial Insight: Without immediate digitization and categorization, individuals struggle to gain real-time visibility into their spending patterns. Small businesses lack granular insights into operational costs, hindering effective financial planning and decision-making.
• Audit Preparedness: In the event of an audit, quickly retrieving and presenting organized expense documentation is crucial. A disorganized pile of paper receipts can turn an audit into a stressful and resource-intensive ordeal.
• Environmental Impact: While minor, the reliance on paper receipts contributes to paper waste. A digital solution aligns with modern environmental consciousness.

The "AI Receipt Scanner" addresses these pain points by offering an intuitive, AI-powered solution to automate the digitization, extraction, and categorization of expense receipts. It aims to empower users to effortlessly manage their expenses, gain clarity on their spending, and maintain impeccable financial records, thereby saving time, money, and reducing stress associated with traditional expense tracking.

2. Solution Overview

The AI Receipt Scanner is a mobile-first application designed to transform the cumbersome process of paper receipt management into a streamlined, digital experience. Its core functionality revolves around leveraging artificial intelligence to process physical receipts captured via a smartphone camera, extracting relevant data, and intelligently categorizing expenses.

User Journey:

1. Capture: The user opens the app and uses their device's camera to take a clear photo of a paper receipt.
2. Process (AI): The captured image is immediately sent for AI processing. Gemini's Vision capabilities perform OCR (Optical Character Recognition) to extract all textual information, followed by further AI analysis to identify key data points like vendor name, date, total amount, and individual line items. Concurrently, the AI intelligently suggests an expense category (e.g., "Food & Dining," "Transportation").
3. Review & Edit: The app presents the extracted data to the user in an editable format. The user can quickly review the AI's extraction and categorization, making any necessary corrections or additions (e.g., adding notes, assigning to a specific project).
4. Store & Sync: Upon confirmation, the processed receipt data (structured text) and the original image are securely stored in the cloud. For offline access and resilience, a local copy is also maintained on the device.
5. Access & Analyze: Users can later access their digital receipts, filter them by category, date, or vendor, and gain insights into their spending habits.

Key Features:

• Camera Input for Receipts: A dedicated, optimized camera interface for capturing clear images of receipts.
• AI Text Extraction (OCR): Powered by Google Gemini Vision API to accurately extract all relevant text and structured data from receipt images.
• Auto-categorization: Intelligent categorization of expenses based on extracted data, utilizing Gemini's natural language understanding capabilities.
• Digital Receipt Storage: Secure cloud storage for both original receipt images and extracted structured data.
• Offline Functionality: Ability to capture and process receipts even without an internet connection, with data syncing once connectivity is restored.
• User Interface for Review & Editing: A clean, intuitive interface to review AI-extracted data, make corrections, and add custom notes or tags.
• Search & Filtering: Robust capabilities to search and filter stored receipts by various criteria (date, category, vendor, amount).

The solution prioritizes ease of use, accuracy, and robust data persistence, aiming to be the go-to tool for hassle-free expense tracking.

3. Architecture & Tech Stack Justification

The architecture for the AI Receipt Scanner is designed for rapid development, scalability, and ease of maintenance, leveraging Google's serverless ecosystem. The "Beginner Difficulty" constraint guides the choice towards managed services that abstract away complex infrastructure concerns.

Overall Architecture Diagram (Conceptual):

[User Device (iOS/Android)] <------> [React Native App]
    |                                   |
    | (1. Capture Image)                | (2. Upload Image)
    |                                   |
[PouchDB (Local Cache)] <-------------> [Firebase Cloud Storage (Images)]
    ^                                   |
    | (7. Sync Data)                    | (3. Cloud Function Trigger)
    |                                   |
[Firebase Firestore (Structured Data)] <-- [Firebase Cloud Functions (Backend Logic)] <--> [Gemini Vision API]
    ^                                                           ^
    | (6. Data Storage)                                         | (4. AI Processing)
    |                                                           | (5. Response Parsing & Categorization)
    ------------------------------------------------------------------------------------------------------

Tech Stack Justification:

1. Frontend: React Native

   • Justification: Allows for a single codebase to target both iOS and Android mobile platforms, significantly accelerating development and reducing maintenance overhead compared to native development. It provides access to native device features (like the camera) while offering a familiar JavaScript/React development experience.
   • "Web Version" Note: While React Native for Web exists, for a true web application companion, a separate React web app sharing design systems and some utility components is often more performant and SEO-friendly. For this "Beginner" project, the focus is primarily mobile, with React Native serving as the core. Reusing some UI components with React Native Web for a dashboard type web interface is feasible but not for a primary web receipt scanning flow.

2. AI Engine: Gemini API (Vision)

   • Justification: Gemini Vision is a powerful multimodal AI model capable of processing images to extract text (OCR) and understand context. Its advanced capabilities are crucial for accurate data extraction from varied receipt formats and for intelligent auto-categorization, significantly outperforming simpler OCR solutions. As Staff AI Engineer at Google, leveraging our internal cutting-edge models ensures best-in-class performance and ease of integration within the Google Cloud ecosystem.
   • Alternatives (and why Gemini is better): Tesseract (open-source, less accurate for varied formats, requires self-hosting), Google Cloud Vision API (excellent, but Gemini Pro Vision is a more integrated, powerful multimodal offering covering both OCR and semantic understanding in one API call).

3. Backend Services: Firebase

   • Justification: Firebase is a comprehensive serverless platform that provides a robust, scalable, and fully managed backend infrastructure, ideal for rapid development and "beginner-friendly" projects.
     • Firebase Authentication: Handles user registration, login, and session management securely and with minimal setup.
     • Firebase Cloud Storage: Provides scalable, secure storage for raw receipt images. It seamlessly integrates with Cloud Functions for event-driven processing.
     • Firebase Firestore: A NoSQL document database offering real-time data synchronization. It's perfect for storing structured receipt data (extracted text, categories, metadata) and user profiles, providing flexible querying and automatic scaling.
     • Firebase Cloud Functions: A serverless compute platform. Critical for securely calling the Gemini API (protecting API keys) and orchestrating backend logic like image preprocessing, data parsing, and saving to Firestore. Functions scale automatically based on demand.

4. Offline Data Store: PouchDB

   • Justification: PouchDB is an in-browser (or Node.js) database designed to work offline. It's a local NoSQL database that is invaluable for building robust offline-first mobile applications.
     • Offline Receipt Capture: Users can take receipt photos and enter details even without an internet connection. PouchDB stores these locally.
     • Data Cache: Caches previously synced receipts, allowing users to view and edit past expenses offline.
     • Sync Strategy: While PouchDB traditionally syncs with CouchDB/Cloudant, for this architecture, it acts as a robust local cache and a temporary store for new receipts that will be eventually pushed to Firebase Firestore. Existing Firestore data can be "synced" to PouchDB for offline access.
     • Alternatives (and why PouchDB is preferred here): SQLite via react-native-sqlite-storage (more complex to manage schemas and migrations, less flexible for unstructured data), AsyncStorage (key-value store, not suitable for complex queries or large datasets). PouchDB offers a more feature-rich, document-oriented approach suitable for our data.

This integrated stack minimizes operational overhead, allows developers to focus on application logic and user experience, and provides a clear path to scaling as the user base grows.

4. Core Feature Implementation Guide

This section outlines the detailed implementation strategy for the core features, including pseudo-code and pipeline designs.

A. Camera Input & Image Handling

The first step is capturing a high-quality image of the receipt and preparing it for AI processing and storage.

Pipeline:

1. Capture: User taps "Scan Receipt," activates camera.
2. Preview & Confirm: User captures image, app displays a preview.
3. Local Storage (PouchDB): If offline, image (or a temporary reference) is stored locally.
4. Preprocessing: Image compression and resizing to optimize upload speed and Gemini API latency/cost.
5. Upload: Upload processed image to Firebase Cloud Storage.

Pseudo-code (React Native):

// Using 'react-native-vision-camera' (or 'expo-camera' for Expo projects)
import { useCameraDevices, Camera } from 'react-native-vision-camera';
import ImageResizer from '@react-native-community/image-resizer';
import { v4 as uuidv4 } from 'uuid'; // For unique local IDs

// ... inside a React Native component
const devices = useCameraDevices('back');
const device = devices.find(d => d.position === 'back');
const cameraRef = useRef(null);

const captureReceipt = async () => {
  if (cameraRef.current == null) return;

  try {
    const photo = await cameraRef.current.takePhoto({
      qualityPrioritization: 'quality', // Prioritize image quality
      flash: 'auto',
    });

    const originalImagePath = photo.path;
    const resizedImage = await ImageResizer.createResizedImage(
      originalImagePath,
      1024, // Max width
      1500, // Max height
      'JPEG', // Format
      80, // Quality (0-100)
      0, // Rotation
      // No target path - writes to cache directory by default
    );

    const localReceiptId = uuidv4();
    // 1. Store in PouchDB for offline resilience and immediate feedback
    await PouchDBInstance.put({
      _id: localReceiptId,
      status: 'pending_upload',
      image_local_path: resizedImage.uri, // Or base64 for small images
      timestamp: new Date().toISOString(),
      // Other metadata
    });

    // 2. Upload to Firebase Storage
    const storageRef = firebase.storage().ref(`receipts/${firebase.auth().currentUser.uid}/${localReceiptId}.jpg`);
    const response = await fetch(resizedImage.uri);
    const blob = await response.blob();
    await storageRef.put(blob);

    const imageUrl = await storageRef.getDownloadURL();

    // 3. Update PouchDB with Cloud Storage URL and status
    const doc = await PouchDBInstance.get(localReceiptId);
    await PouchDBInstance.put({
      ...doc,
      status: 'uploaded',
      image_cloud_url: imageUrl,
    });

    // Proceed to AI processing via Cloud Function (triggered by upload)
    console.log('Image uploaded:', imageUrl);

  } catch (error) {
    console.error('Error capturing or uploading receipt:', error);
    // Handle error, inform user
  }
};

return (
  <Camera
    ref={cameraRef}
    style={StyleSheet.absoluteFill}
    device={device}
    isActive={true}
    photo={true}
  />
  // ... button to trigger captureReceipt
);

B. AI Text Extraction (OCR) & Data Parsing Pipeline

This is the core AI-driven step, transforming a raw image into structured, actionable data.

Pipeline:

1. Trigger: Firebase Cloud Storage upload event (from step A) triggers a Firebase Cloud Function.
2. Fetch Image: Cloud Function securely fetches the uploaded image from Cloud Storage.
3. Call Gemini API: Cloud Function constructs a request to the Gemini Vision API with the image and a specific prompt.
4. Receive Response: Gemini API returns extracted text and potentially structured data.
5. Parse & Normalize: Cloud Function parses Gemini's response (which we instruct to be JSON) and normalizes the data structure.
6. Store in Firestore: The extracted and normalized data is saved to Firestore, linked to the user and the original image URL.

Pseudo-code (Firebase Cloud Function - onNewReceiptImageUpload):

import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin';
import { GoogleGenerativeAI } from '@google/generative-ai'; // Gemini client library

admin.initializeApp();
const db = admin.firestore();
const genAI = new GoogleGenerativeAI(functions.config().gemini.api_key); // Secure API key via Cloud Functions config
const model = genAI.getGenerativeModel({ model: 'gemini-pro-vision' });

export const processReceiptImage = functions.storage.object().onFinalize(async (object) => {
  const fileBucket = object.bucket; // The Storage bucket that contains the file.
  const filePath = object.name; // File path in the bucket.
  const contentType = object.contentType; // File content type.
  const resourceState = object.resourceState; // The resourceState is 'exists' or 'not_exists' (for file deletion).

  // Exit if this is a deletion or not an image.
  if (resourceState === 'not_exists' || !contentType.startsWith('image/')) {
    console.log('This is not an image or is a deletion.');
    return null;
  }

  // Extract user ID and receipt ID from filePath (e.g., receipts/{userId}/{receiptId}.jpg)
  const pathSegments = filePath.split('/');
  if (pathSegments.length < 3 || pathSegments[0] !== 'receipts') {
    console.error('Invalid file path format:', filePath);
    return null;
  }
  const userId = pathSegments[1];
  const receiptId = pathSegments[2].split('.')[0]; // Remove .jpg extension

  const bucket = admin.storage().bucket(fileBucket);
  const file = bucket.file(filePath);

  // Download image buffer to send to Gemini
  const imageBuffer = await file.download();
  const imageBase64 = imageBuffer[0].toString('base64');

  const prompt = `You are an expert financial assistant. Analyze this image of a receipt and extract the following information. Present the output as a JSON object. Ensure numerical values are parsed correctly as floats and dates as ISO 8601 strings (YYYY-MM-DD). If a field is not found, use null or an empty array for lists.
  {
    "store_name": "<text>",
    "transaction_date": "<YYYY-MM-DD>",
    "transaction_time": "<HH:MM:SS, optional>",
    "currency": "<e.g., USD, EUR>",
    "items": [
      {"description": "<text>", "quantity": <float>, "unit_price": <float>, "item_total": <float>},
      ...
    ],
    "subtotal": <float>,
    "tax": <float>,
    "tip": <float>,
    "total_amount": <float>,
    "payment_method": "<text, e.g., Credit Card, Cash>"
  }`;

  try {
    const result = await model.generateContent([
      prompt,
      {
        inlineData: {
          mimeType: contentType,
          data: imageBase64,
        },
      },
    ]);

    const geminiResponseText = result.response.text();
    let extractedData;
    try {
      // Gemini might wrap JSON in markdown code block
      const jsonMatch = geminiResponseText.match(/```json\n([\s\S]*?)\n```/);
      if (jsonMatch && jsonMatch[1]) {
        extractedData = JSON.parse(jsonMatch[1]);
      } else {
        extractedData = JSON.parse(geminiResponseText);
      }
    } catch (parseError) {
      console.error('Failed to parse Gemini JSON response:', parseError, 'Raw:', geminiResponseText);
      // Attempt simpler text extraction if JSON fails or fallback
      extractedData = { raw_text_fallback: geminiResponseText };
    }

    // Initialize document data
    let receiptDoc = {
      userId: userId,
      imageId: receiptId,
      imageUrl: `gs://${fileBucket}/${filePath}`, // Google Cloud Storage URI
      extractedAt: admin.firestore.FieldValue.serverTimestamp(),
      status: 'extracted',
      extractedData: extractedData,
      // Placeholder for category (will be set in the next step)
      category: 'Uncategorized',
      totalAmount: extractedData?.total_amount || 0,
      transactionDate: extractedData?.transaction_date ? new Date(extractedData.transaction_date) : null,
      storeName: extractedData?.store_name || 'Unknown Vendor',
      // Add a field for easy PouchDB syncing if needed
      _rev: 1, // For PouchDB like versioning
    };

    // Store raw extracted data in Firestore
    await db.collection('receipts').doc(receiptId).set(receiptDoc, { merge: true });

    console.log(`Receipt ${receiptId} processed for user ${userId}.`);
    return null;

  } catch (error) {
    console.error(`Error processing receipt ${receiptId}:`, error);
    // Update receipt status to 'failed_processing' in Firestore/PouchDB
    await db.collection('receipts').doc(receiptId).set({
      status: 'failed_processing',
      errorMessage: error.message,
    }, { merge: true });
    return null;
  }
});

C. Auto-Categorization (AI)

This step leverages Gemini's understanding to assign a suitable category to the expense.

Pipeline:

1. Trigger: Can be part of the same Cloud Function after OCR, or a separate Cloud Function triggered by a new receipt being saved to Firestore.
2. Call Gemini API: Cloud Function calls Gemini with the extracted text data and a predefined list of categories.
3. Receive Category: Gemini returns the most appropriate category.
4. Update Firestore: The assigned category is updated in the Firestore document.

Pre-defined Categories (Example): ["Food & Dining", "Groceries", "Transportation", "Utilities", "Rent/Mortgage", "Shopping", "Entertainment", "Health", "Education", "Travel", "Business Expenses", "Miscellaneous"]

Pseudo-code (Continuation of Cloud Function or new categorizeReceipt function):

// ... inside processReceiptImage after initial receiptDoc is created
// Add a function to categorize the receipt
const categorizeExpense = async (extractedReceiptData) => {
  const categorizationModel = genAI.getGenerativeModel({ model: 'gemini-pro' }); // Using text-only model for categorization
  const categories = ["Food & Dining", "Groceries", "Transportation", "Utilities", "Rent/Mortgage", "Shopping", "Entertainment", "Health", "Education", "Travel", "Business Expenses", "Miscellaneous"];

  const categorizationPrompt = `Given the following receipt details (store name, items, total amount):
  Store: ${extractedReceiptData?.store_name || 'N/A'}
  Items: ${extractedReceiptData?.items?.map(item => item.description).join(', ') || 'N/A'}
  Total: ${extractedReceiptData?.total_amount || 'N/A'}

  Categorize this expense into one of the following categories: ${JSON.stringify(categories)}.
  Provide only the most appropriate category name as a single string. If unsure or if the context is ambiguous, default to "Miscellaneous".`;

  try {
    const result = await categorizationModel.generateContent(categorizationPrompt);
    const category = result.response.text().trim();
    if (categories.includes(category)) { // Basic validation
      return category;
    }
    return "Miscellaneous"; // Fallback
  } catch (error) {
    console.error('Error during categorization:', error);
    return "Miscellaneous"; // Fallback on error
  }
};

// ... inside processReceiptImage, after `await db.collection('receipts').doc(receiptId).set(receiptDoc, { merge: true });`
const category = await categorizeExpense(extractedData);
await db.collection('receipts').doc(receiptId).update({
  category: category,
  status: 'categorized', // Update status
});
console.log(`Receipt ${receiptId} categorized as ${category}.`);

D. Digital Receipt Storage & Sync

Ensuring data persistence, integrity, and offline accessibility.

• Firebase Cloud Storage: Stores the original (resized) receipt images. Access is secured by Firebase Storage Rules, ensuring only authenticated users can read/write their own images.
• Firebase Firestore: Stores the structured, AI-extracted, and user-edited receipt data.
  • Data Model (Example for Firestore document receipts/{receiptId}):

    {
      "receiptId": "unique-uuid-for-receipt",
      "userId": "firebase-auth-uid",
      "imageUrl": "https://firebasestorage.googleapis.com/...",
      "thumbnailUrl": "https://...", // (Optional: smaller version for lists)
      "storeName": "Starbucks",
      "transactionDate": "2023-10-26T00:00:00Z", // Firestore Timestamp
      "totalAmount": 5.75,
      "category": "Food & Dining",
      "notes": "Coffee & muffin with Sarah",
      "extractedData": { // Raw data from Gemini for debugging/reference
        "store_name": "Starbucks",
        "transaction_date": "2023-10-26",
        "items": [...],
        // ... full Gemini output
      },
      "createdAt": "2023-10-26T10:30:00Z", // Server Timestamp
      "lastEditedAt": "2023-10-26T11:00:00Z",
      "status": "categorized" // pending_upload, extracted, categorized, edited, failed
    }
    

• PouchDB (Offline): Serves as the primary local storage for receipts.
  • Strategy:
    • New Receipts: Receipts captured offline are stored in PouchDB (status: 'pending_upload'). When online, they are synced to Firebase Storage and processed, then updated in PouchDB.
    • Existing Receipts: When the app is online, it can fetch a user's recent receipts from Firestore and cache them in PouchDB for offline viewing/editing. Changes made offline are marked and synced back to Firestore when connectivity is restored.

Pseudo-code (React Native - PouchDB / Firestore Sync):

// PouchDB setup
import PouchDB from 'pouchdb-react-native'; // Use the react-native specific build
const PouchDBInstance = new PouchDB('receipts_db');

// Function to save/update receipt locally
const saveReceiptLocally = async (receiptData) => {
  try {
    const doc = { ...receiptData, _id: receiptData.receiptId || uuidv4() };
    const response = await PouchDBInstance.put(doc);
    return response;
  } catch (err) {
    if (err.status === 409) { // Document already exists, update it
      const existingDoc = await PouchDBInstance.get(receiptData.receiptId);
      const response = await PouchDBInstance.put({ ...existingDoc, ...receiptData, _rev: existingDoc._rev });
      return response;
    }
    console.error("Error saving to PouchDB:", err);
    throw err;
  }
};

// Function to sync Firestore to PouchDB (for offline viewing)
const syncFirestoreToPouchDB = async (userId) => {
  const querySnapshot = await firebase.firestore()
    .collection('receipts')
    .where('userId', '==', userId)
    .orderBy('transactionDate', 'desc')
    .limit(100) // Fetch recent receipts
    .get();

  const receiptsToSync = [];
  querySnapshot.forEach(doc => {
    receiptsToSync.push({ ...doc.data(), _id: doc.id }); // PouchDB uses _id
  });

  try {
    // Bulk put/update into PouchDB
    await PouchDBInstance.bulkDocs(receiptsToSync.map(r => ({ ...r, _id: r.receiptId, _rev: r._rev || undefined })));
    console.log('Firestore receipts synced to PouchDB.');
  } catch (error) {
    console.error('Error syncing Firestore to PouchDB:', error);
  }
};

// Function to push local changes to Firestore (e.g., user edits offline)
const pushLocalChangesToFirestore = async (receiptId) => {
  try {
    const localReceipt = await PouchDBInstance.get(receiptId);
    if (localReceipt.status === 'edited_offline' || localReceipt.status === 'pending_upload') {
      // Logic to upload image if pending_upload, then save data
      const docRef = firebase.firestore().collection('receipts').doc(receiptId);
      await docRef.set({
        ...localReceipt,
        lastEditedAt: firebase.firestore.FieldValue.serverTimestamp(),
        status: 'synced',
        _id: firebase.firestore.FieldValue.delete(), // Remove PouchDB specific _id
        _rev: firebase.firestore.FieldValue.delete(), // Remove PouchDB specific _rev
      }, { merge: true });

      // Update local PouchDB doc to reflect synced status
      await PouchDBInstance.put({ ...localReceipt, _rev: (await PouchDBInstance.get(receiptId))._rev, status: 'synced' });
      console.log(`Receipt ${receiptId} synced from PouchDB to Firestore.`);
    }
  } catch (error) {
    console.error('Error pushing local changes to Firestore:', error);
  }
};

E. User Interface (Review & Edit)

A critical step for user control and data accuracy.

• Display Extracted Data: Present the AI-extracted storeName, transactionDate, totalAmount, category, and items in clear, editable fields.
• Original Image View: Allow users to toggle between the extracted data and the original receipt image for verification.
• Correction & Input: Provide input fields for users to correct OCR errors, change the auto-categorized category (e.g., via a dropdown), and add custom notes or tags.
• Save: A clear "Save" or "Confirm" button to commit changes to Firestore (and update PouchDB).

Pseudo-code (React Native Component):

import React, { useState, useEffect } from 'react';
import { View, Text, TextInput, Button, Image, ScrollView, StyleSheet } from 'react-native';
import { Picker } from '@react-native-picker/picker'; // For category selection

const EditReceiptScreen = ({ route, navigation }) => {
  const { receiptId } = route.params;
  const [receipt, setReceipt] = useState(null);
  const [editedData, setEditedData] = useState(null);
  const categories = ["Food & Dining", "Groceries", "Transportation", "Utilities", /* ... */ "Miscellaneous"];

  useEffect(() => {
    const fetchReceipt = async () => {
      // Fetch from PouchDB first for speed, fallback to Firestore
      try {
        const localDoc = await PouchDBInstance.get(receiptId);
        setReceipt(localDoc);
        setEditedData(localDoc);
      } catch (e) {
        // Not found locally, fetch from Firestore
        const doc = await firebase.firestore().collection('receipts').doc(receiptId).get();
        if (doc.exists) {
          const firestoreData = { ...doc.data(), receiptId: doc.id };
          setReceipt(firestoreData);
          setEditedData(firestoreData);
          await saveReceiptLocally(firestoreData); // Cache in PouchDB
        }
      }
    };
    fetchReceipt();
  }, [receiptId]);

  const handleInputChange = (field, value) => {
    setEditedData(prev => ({ ...prev, [field]: value }));
  };

  const saveChanges = async () => {
    if (!editedData) return;
    try {
      // Update Firestore
      await firebase.firestore().collection('receipts').doc(receiptId).update({
        storeName: editedData.storeName,
        transactionDate: new Date(editedData.transactionDate),
        totalAmount: parseFloat(editedData.totalAmount),
        category: editedData.category,
        notes: editedData.notes,
        lastEditedAt: firebase.firestore.FieldValue.serverTimestamp(),
        status: 'edited',
      });
      // Update PouchDB with new status and data
      await saveReceiptLocally({ ...editedData, status: 'synced' });
      navigation.goBack();
      // Inform user of success
    } catch (error) {
      console.error('Error saving receipt changes:', error);
      // Inform user of error
    }
  };

  if (!receipt) {
    return <Text>Loading receipt...</Text>;
  }

  return (
    <ScrollView style={styles.container}>
      <Text style={styles.label}>Original Receipt Image:</Text>
      <Image source={{ uri: receipt.imageUrl }} style={styles.receiptImage} />

      <Text style={styles.label}>Store Name:</Text>
      <TextInput
        style={styles.input}
        value={editedData?.storeName}
        onChangeText={(text) => handleInputChange('storeName', text)}
      />

      <Text style={styles.label}>Total Amount:</Text>
      <TextInput
        style={styles.input}
        value={String(editedData?.totalAmount)}
        onChangeText={(text) => handleInputChange('totalAmount', text)}
        keyboardType="numeric"
      />

      <Text style={styles.label}>Category:</Text>
      <Picker
        selectedValue={editedData?.category}
        onValueChange={(itemValue) => handleInputChange('category', itemValue)}
        style={styles.picker}
      >
        {categories.map((cat, index) => (
          <Picker.Item key={index} label={cat} value={cat} />
        ))}
      </Picker>

      <Text style={styles.label}>Notes:</Text>
      <TextInput
        style={styles.input}
        value={editedData?.notes}
        onChangeText={(text) => handleInputChange('notes', text)}
        multiline
      />

      <Button title="Save Changes" onPress={saveChanges} />
    </ScrollView>
  );
};

const styles = StyleSheet.create({
  // ... basic styles for inputs, labels, image
  receiptImage: { width: '100%', height: 300, resizeMode: 'contain', marginBottom: 15 },
  input: { borderWidth: 1, borderColor: '#ccc', padding: 10, marginBottom: 10 },
  picker: { borderWidth: 1, borderColor: '#ccc', marginBottom: 10 },
});
export default EditReceiptScreen;

5. Gemini Prompting Strategy

The quality of AI extraction and categorization is directly proportional to the effectiveness of the prompts given to the Gemini API. Here, we outline the strategies for crafting robust prompts.

Core Principles:

1. Be Explicit: Clearly state the task, desired output format, and expected information.
2. Provide Context: Inform Gemini of its role (e.g., "expert financial assistant").
3. Define Constraints: Specify acceptable values, formats, and fallback options.
4. Structure Output: Request JSON for structured data extraction, making parsing deterministic.
5. Iterative Refinement: Start simple, then add complexity and examples as needed.

A. OCR Prompt (Initial Data Extraction)

Goal: Accurately extract all critical, structured data points from a receipt image. Gemini Model: gemini-pro-vision (for multimodal input: image + text prompt)

You are an expert financial assistant tasked with accurately extracting information from receipt images.
Analyze the provided receipt image and extract the following details. Your output MUST be a JSON object, conforming strictly to the specified schema.
If a field is not present or cannot be confidently identified, return `null` for individual values, or an empty array `[]` for lists (e.g., `items`).
Ensure numerical values are parsed correctly as floating-point numbers and dates are in ISO 8601 `YYYY-MM-DD` format. Times should be `HH:MM` or `HH:MM:SS`.

JSON Schema:
{
  "store_name": "<string | null>",
  "transaction_date": "<YYYY-MM-DD | null>",
  "transaction_time": "<HH:MM:SS | HH:MM | null>",
  "currency": "<string, e.g., USD, EUR, GBP | null>",
  "items": [
    {
      "description": "<string | null>",
      "quantity": "<float | null>",
      "unit_price": "<float | null>",
      "item_total": "<float | null>"
    }
  ],
  "subtotal": "<float | null>",
  "tax": "<float | null>",
  "tip": "<float | null>",
  "total_amount": "<float | null>",
  "payment_method": "<string, e.g., Credit Card, Cash, Debit | null>",
  "raw_text_extracted": "<string, all visible text for fallback/context | null>"
}

Example of expected output structure (if all fields are found):
```json
{
  "store_name": "Cafe Blossom",
  "transaction_date": "2023-10-27",
  "transaction_time": "14:35:10",
  "currency": "USD",
  "items": [
    {"description": "Espresso", "quantity": 2.0, "unit_price": 3.50, "item_total": 7.00},
    {"description": "Croissant", "quantity": 1.0, "unit_price": 4.25, "item_total": 4.25}
  ],
  "subtotal": 11.25,
  "tax": 0.90,
  "tip": 2.00,
  "total_amount": 14.15,
  "payment_method": "Credit Card",
  "raw_text_extracted": "Cafe Blossom\n123 Main St\nTel: 555-1234\n..."
}

**Reasoning:**
*   **Role & Task:** "Expert financial assistant," "accurately extracting information."
*   **Strict JSON:** Enforces a predictable output for programmatic parsing.
*   **Null Handling:** Explicitly tells Gemini what to do with missing data, preventing arbitrary strings or empty fields.
*   **Data Types:** Guides Gemini on parsing numbers and dates correctly.
*   **`raw_text_extracted`:** Provides a fallback if structured parsing fails or if additional context is needed for debugging.

### B. Categorization Prompt

**Goal:** Assign a precise expense category based on the extracted receipt data.
**Gemini Model:** `gemini-pro` (for text-only input: structured receipt data + text prompt)

You are an intelligent expense categorizer. Your task is to assign the most appropriate category to a financial transaction based on its details.

Here is the extracted information from a receipt: Store Name: "{{extractedData.store_name}}" Items Purchased: "{{extractedData.items | map_description_and_total | join_with_comma}}" Total Amount: {{extractedData.total_amount}} {{extractedData.currency}}

Please select the SINGLE BEST category from the following predefined list: ["Food & Dining", "Groceries", "Transportation", "Utilities", "Rent/Mortgage", "Shopping", "Entertainment", "Health", "Education", "Travel", "Business Expenses", "Miscellaneous", "Personal Care", "Gifts", "Pet Supplies"]

Respond ONLY with the selected category name as a plain string. Do NOT include any additional text, explanations, or formatting (like quotes or JSON). If the category is genuinely uncertain, provide "Miscellaneous".

Example: If Store Name is "Starbucks" and Items are "Coffee, Muffin", the category should be "Food & Dining". Example: If Store Name is "Safeway" and Items are "Milk, Bread, Eggs", the category should be "Groceries". Example: If Store Name is "Uber" and Items are "Ride", the category should be "Transportation".

**Reasoning:**
*   **Role & Task:** "Intelligent expense categorizer," "assign the most appropriate category."
*   **Clear Inputs:** Uses placeholders for extracted data, clearly showing what information Gemini receives.
*   **Predefined List:** Constrains Gemini to a specific set of categories, crucial for consistent data.
*   **Single String Output:** Simplifies parsing, avoids complex response structures.
*   **"Miscellaneous" Fallback:** Provides a default for ambiguous cases.
*   **Few-shot Examples:** Simple examples help guide Gemini to the desired behavior, especially for common scenarios. (The `map_description_and_total` and `join_with_comma` are pseudo-functions here, in actual implementation you'd format the item string in your Cloud Function before sending to Gemini).

### C. Iterative Refinement Strategy

1.  **Initial Testing with Diverse Receipts:** Test prompts against a wide variety of receipt types (different layouts, stores, conditions).
2.  **Error Analysis:** Systematically review cases where Gemini fails (incorrect extraction, wrong category).
3.  **Prompt Tuning:**
    *   For OCR: If data is consistently missed, add specific phrases to the prompt (e.g., "Explicitly look for 'TAX' or 'VAT' fields"). If parsing fails, refine the JSON schema or add more strict formatting requirements.
    *   For Categorization: If categories are wrong, add more few-shot examples that highlight the distinction, or refine the category list itself.
4.  **Safety & Guardrails:** Monitor for unexpected or inappropriate outputs. Gemini models have built-in safety features, but application-specific guardrails might be needed.
5.  **User Feedback Loop:** Incorporate user corrections from the "Review & Edit" screen to identify common AI errors and continuously improve prompts. This could be done by logging corrected data and periodically retraining/tuning prompts based on aggregated corrections.

By meticulously crafting and refining these prompts, the AI Receipt Scanner can achieve high accuracy and provide significant value to users.

---

## 6. Deployment & Scaling

Leveraging the Google Cloud and Firebase ecosystem makes deployment and scaling largely automated and straightforward, even for a beginner project.

### A. Development Environment & CI/CD

*   **Local Development:** React Native CLI for local development, device/simulator testing. Firebase Emulators for local testing of Cloud Functions, Firestore, and Storage.
*   **Version Control:** Git (e.g., GitHub, GitLab, Cloud Source Repositories).
*   **CI/CD (Continuous Integration/Continuous Deployment):**
    *   **Mobile:** Google Cloud Build can automate React Native app builds (iOS/Android) on every push to a `main` branch. This includes running tests, building release APKs/AABs for Android, and IPA/archives for iOS, then publishing to Google Play Console and App Store Connect (requires initial manual setup and credentials).
    *   **Backend (Cloud Functions):** Cloud Functions are automatically deployed via `firebase deploy --only functions` from a CI/CD pipeline, or directly from a local machine for rapid iteration.
    *   **Firebase Hosting:** For any web-based components (like a marketing page or a potential React Native Web dashboard), Firebase Hosting provides fast, secure, and globally distributed static asset hosting.

### B. Core Services Deployment & Scaling

1.  **Firebase Authentication:**
    *   **Deployment:** Simply configured in the Firebase console (enable desired providers like Email/Password, Google Sign-in).
    *   **Scaling:** Fully managed by Firebase, automatically scales to millions of users without intervention.

2.  **Firebase Cloud Storage (for Receipt Images):**
    *   **Deployment:** No explicit deployment needed beyond creating a Firebase project.
    *   **Scaling:** Automatically scales to store petabytes of data and handle massive access patterns. Data is globally replicated for high availability and low latency.
    *   **Security:** Enforced via Firebase Storage Security Rules, configured in the Firebase console, limiting access based on user authentication and data ownership (`request.auth.uid == resource.metadata.ownerId`).

3.  **Firebase Firestore (for Structured Receipt Data):**
    *   **Deployment:** No explicit deployment; databases are provisioned on project creation. Data model is schema-less.
    *   **Scaling:** Horizontally scales to handle large datasets and high concurrent read/write operations. Performance relies on efficient data modeling and proper indexing (automatic for simple queries, custom for complex ones).
    *   **Security:** Enforced via Firebase Firestore Security Rules, configured in the Firebase console, defining read/write access based on user authentication and data ownership.

4.  **Firebase Cloud Functions (Backend Logic & Gemini Orchestration):**
    *   **Deployment:** Deployed using the Firebase CLI (`firebase deploy --only functions`).
    *   **Scaling:** Serverless, scales automatically from zero to thousands of instances per second based on incoming events (e.g., Cloud Storage triggers, HTTP requests).
    *   **Concurrency:** Can be configured to handle multiple requests concurrently per instance.
    *   **Cost:** Pay-per-invocation, compute time, and outbound network traffic.

5.  **Gemini API:**
    *   **Deployment:** No deployment needed. It's a managed API.
    *   **Scaling:** Designed for high throughput and low latency.
    *   **Quotas:** Monitor API usage against project quotas in Google Cloud Console. Request quota increases if necessary as user base grows.
    *   **Security:** API keys should *never* be exposed client-side. Using Cloud Functions acts as a secure proxy, protecting the API key and enabling server-side rate limiting/usage tracking.

### C. Monitoring, Logging & Error Handling

*   **Firebase Crashlytics:** For real-time crash reporting and issue tracking in the React Native app.
*   **Google Cloud Logging:** Cloud Functions logs automatically stream to Cloud Logging. Set up log-based metrics and alerts for errors, high latency, or specific events.
*   **Google Cloud Monitoring:** Dashboards and alerts for Cloud Function invocations, errors, execution times, Firestore read/write operations, Storage usage, and Gemini API usage/errors.
*   **Error Handling:** Implement robust `try-catch` blocks in all asynchronous operations (both client and server-side). Display user-friendly error messages in the app.
*   **Retries:** Implement exponential backoff for API calls (especially to external services like Gemini) to handle transient errors.

### D. Security Considerations

*   **Firebase Security Rules:** Absolutely critical for protecting Firestore and Cloud Storage data. Ensure users can only read/write their own data (`request.auth.uid == resource.data.userId`).
*   **API Key Management:** Gemini API keys must be securely stored and accessed only from trusted server environments (i.e., Firebase Cloud Functions), never directly in the client-side code. Use `functions.config()` for sensitive values.
*   **Authentication:** Rely on Firebase Auth for robust and secure user authentication.
*   **Input Validation:** Validate user input on the client-side and, more importantly, on the server-side (in Cloud Functions) to prevent malicious data or malformed requests.
*   **Image Security:** While Firebase Storage encrypts data at rest, ensure no sensitive personal information (beyond what's on a receipt) is inadvertently captured or stored.

This comprehensive deployment and scaling strategy ensures that the AI Receipt Scanner can grow from a beginner project to a robust, production-ready application capable of handling increasing user demand while maintaining high performance and security.