Annota – Scan & Learn Japanese from Any Book

Annota is a lightweight web app that helps professionals and learners study Japanese directly from their physical books. By combining OCR, contextual annotations, and a simple mobile-first experience, Annota reduces the friction of jumping between printed material, dictionaries, and online references especially for users learning Japanese for work.

This project started as part of the Gemini 3 Hackathon – Scan & Learn Japanese, with a clear focus: validate whether people find it valuable to learn Japanese from photographed book content using OCR and contextual explanations.

People Behind The Annota

This project was built by a small cross-functional team with complementary strengths:

1. Yosep Novento N – Software Engineer (Backend)

Yosep focuses on building a robust, scalable backend and ensuring smooth integration between the OCR engine, Gemini, and the web app. He is responsible for processing uploaded images, handling authentication and data storage, and turning raw image input into reliable, structured text and annotations that Annota can work with.

1. Naufaldi Rafif S – Software Engineer (Frontend)

Naufaldi leads the frontend implementation and brings the product experience to life in the browser. He works on the mobile-first interface, page capture and upload flows, text highlighting interactions, and presenting annotation results in a fast, responsive way so users can scan, tap, and learn without friction.

1. Anggita Satriya K – Product Designer

Anggita is responsible for the end-to-end product experience, from defining the core “scan–read–tap–learn” journey to crafting the visual design. He maps user needs, prioritizes features for the MVP, and ensures Annota feels simple and focused for professionals learning Japanese from physical books.

This core team is strongly supported by Denny Alfarisi, author of Shigoto Japan. His book serves as the primary reference and data source for shaping Annota’s learning context. Shigoto Japan is used as the main training material for Gemini, so that the AI can learn work-related Japanese and generate explanations, translations, and examples grounded in real professional scenarios. This collaboration helps Annota move beyond generic translation and deliver annotations that are practical, contextual, and relevant for learners using Japanese in their jobs.

For many Japanese learners, especially those using work-related materials (like Shigoto Japan), the most natural way to learn is still through physical books. But that comes with familiar pain points:

1. Learners constantly switch between the book, dictionary apps, and browser tabs just to understand a single sentence.
2. Looking up unfamiliar kanji or phrases is slow, especially for vertical or dense text layouts.
3. Notes, translations, and explanations are scattered across screenshots, notebooks, or different apps, making it hard to revisit what they’ve learned.
4. Most existing tools are optimized for generic translation, not professional context or repeatable learning from the same materials.

In short, there was no simple way to: take a photo of a page, turn it into text, tap on a phrase, and immediately get contextual explanations tailored for work situations then come back to those insights later.

Annota was designed to close that gap with a focused, MVP-first approach.

The Objective

The primary objective of the MVP was to validate the value of learning Japanese directly from photographed book content using OCR and contextual annotations.

More concretely, we aimed to:

1. Enable fast conversion of book images into readable digital text.
2. Help users understand Japanese words or sentences in a professional/work context.
3. Allow users to save and revisit key annotations as part of their learning journey.
4. Deliver a usable, testable MVP in two weeks, suitable for a hackathon timeline.

Product Design

Core Experience

The core Annota experience is designed as a mobile-first, "scan read tap learn" loop:

1. App Entry & Sign-In

Users enter via a simple landing page and authenticate with Google OAuth. No extra accounts, no friction.

1. Capture or Upload

Users can:

1. Take a photo of a book page directly from the camera, or
2. Upload an existing photo from their gallery.
3. OCR Result Preview

The system converts the image into Japanese text. Users see a clean text view mapped from their page, and can scroll, skim, or focus on sections they don’t understand.

1. Highlight & Annotation

Users tap and drag to highlight a word or sentence. Annota then displays:

1. Meaning: direct translation.
2. Usage example: sentences in professional/work context.
3. Usage timing: when and in what situation the phrase is used.
4. Word breakdown: explanation of each word in a sentence.
5. Alternative meaning: different interpretations across fields or contexts.
6. Bookmark & History

Users can:

1. Bookmark specific annotations they want to revisit.
2. Access Scan History to reopen previously uploaded pages and continue learning without starting from scratch.
3. Language Preference

Explanations default to English, but users can adjust their preferred explanation language so the learning feels more comfortable.

Technology & Constraints

While the PRD leaves some implementation details open (e.g., exact OCR engine selection), it sets several important constraints and considerations:

1. OCR Engine

Evaluating options such as Google Vision OCR or Open Tesseract for Japanese text recognition, including partial support for vertical text layouts.

1. Annotation Generation

Exploring whether annotations will be fully AI-generated via LLMs with prompt constraints or supported by rule-based systems. For the MVP, the focus is on achieving reliable, fast responses rather than full linguistic coverage.

1. Performance Targets
2. Annotation response time: ≤ 3 seconds.
3. OCR success rate: ≥ 85% for uploaded images.
4. Optimized specifically for mobile web.
5. Security & Reliability
6. OAuth-based authentication for sign-in.
7. Saved data persists across sessions for bookmarks and history.
8. Upload size and storage policy (e.g., 5–10 MB per image, default persistence unless deleted) are defined as open questions to refine post-MVP.

These constraints help balance hackathon speed with a realistic path to production-ready quality.