1 / ARCHITECTURE
Basic architecture of business AI
Whether handling customer enquiries or processing forms, business AI follows the same five-stage structure. Only the “AI decision” stage is delegated to the model; the surrounding stages are handled by reliable programs and human review.
- 1Input
Email, forms, PDFs, recordings, chat — the point where a task begins. Here we identify the type and volume of inputs and how errors mix in.
- 2Data preparation and retrieval
Transcription, document parsing, and field structuring. When referencing internal documents, only the relevant sections are retrieved from a search layer (RAG) with access controls applied.
- 3AI decision
Classification, extraction, summarisation, and generating response drafts or registration candidates. The model handles only this stage, and the reasoning behind each decision is included in the output.
- 4Integration with existing systems
Writing to accounting, sales management, task management, and chat. This is handled by a program with input validation and the appropriate permissions — the model never writes directly.
- 5Human review and records
A responsible person approves and finalises. Uncertain cases are escalated to a human, and a record of the input, output, references, and decision is retained.
Applied examples
Applying the same five stages to specific tasks gives the following. Components are chosen based on the target task, existing systems, and data handling requirements.
| Task | Input | Data preparation and retrieval | AI decision | Integration with existing systems | Human review and records |
|---|---|---|---|---|---|
| Customer enquiry handling | Email / form / chat | Search past responses, FAQs, and product documentation | Classify and generate a response draft | Transfer to correspondence records | Staff approve and send |
| Form and document processing | PDF / scanned image | Parse document fields into structured data | Match against master data and create registration candidates | Register in accounting or sales management system | Review only differences and exceptions |
| Meeting minutes | Meeting recording | Transcribe and organise speakers | Extract summary, decisions, and tasks | Create tickets in task management tool | Staff revise and finalise |
| Internal document search | Chat (Teams / Slack) | Search documents with access controls | Generate a response citing its sources | Return the answer with source links | If evidence is insufficient, respond “I don’t know” and escalate to a human |
2 / DESIGN
Designing RAG and AI agents
First, decide whether to build as a workflow or an agent. For tasks that reference internal documents, design the search layer (RAG) first; for tasks that operate on existing systems, design the agent with the following points established in advance.
Workflow or agent
- Fixed-process tasksStructured as a workflow with defined branches and processing steps. Results are reproducible, and maintenance and costs are predictable.
- Tasks where decisions vary by contextStructured as an agent that selects and calls tools as needed. The available tools, call limits, and stopping conditions are defined in advance.
- Mixed tasksA workflow provides the backbone, with agents placed only at stages that require judgement. Only as much complexity as needed is introduced.
Answers that trace back to their source
- Document managementDefine the scope, freshness, and update responsibilities for searchable documents. Exclude old versions and drafts from search.
- Access controlsOnly documents the user is permitted to view are searched. Permissions are inherited from the source system and not managed separately on the AI side.
- Response formatAlways cite the source. When evidence is insufficient, respond “I don’t know” and escalate to a human.
Limiting scope through structure
- Tool definitionsSeparate read and write operations; attach input validation to writes. Executable operations are restricted to an explicit list.
- Retries and stoppingDefine the retry count on failure, conditions for stopping mid-task, and who handles the escalation.
- Intermediate loggingRecord which tools were called in which order and the reasoning behind each decision, so the process can be traced after the fact.
3 / EVALUATION
Evaluation and quality control
“Works” and “is usable” are not the same. Build an evaluation set representative of real work and a set of pass criteria first, then apply the same standard consistently from prototyping through production operation.
Measure with inputs representative of real work
- How to build itCollect typical cases, edge cases, and failure cases from real inputs. What accumulates during the first 45 days of prototyping becomes the foundation.
- Pass criteriaDefine numeric targets per task type — such as “the proportion usable without human correction” and “coverage of critical fields that must not be missed”.
Don’t reduce quality to a single number
- AccuracyWhether the output is correct. Items with high error impact — amounts, dates — are measured individually.
- CoverageWhether required fields or points are missing.
- Review time and effortTime staff spend reviewing and correcting output. This is where adoption impact is mainly measured.
Measure by the same standard after every change
- On changeAfter updating the model, prompt, or search scope, re-evaluate against the same evaluation set before switching to production.
- From production logsAdd cases that staff corrected to the evaluation set and update the criteria.
4 / OPERATION
Permissions, logs and human approval
Limit what AI can do through permissions, place human approval before high-impact operations, and keep a record of all activity. Make it a condition of operation that staff can trace the cause, stop the process, and hand over when a problem occurs.
Define what AI can touch in advance
- Separating reads and writesWrites are restricted in scope and conditions; items with high impact — amounts, counterparties — are finalised by a human.
- Per-user permissionsAI actions are limited to the scope of permissions the individual user holds.
Keep humans in final control
- Where approval sitsHuman approval is required before sending to external parties, registering in a system, and operations involving payments.
- Stopping and handoverDocument how to stop the system and who handles the case afterwards.
Enable tracing the process after the fact
- What to retainInput, output, referenced documents, reasoning, and human corrections.
- How to use themUsed to identify the cause when a problem occurs, and to update the evaluation set. Retention periods are set per information type.
5 / MODELS AND PLATFORMS
How we choose models and platforms
Select by role rather than locking into a specific product. Prioritise environments the organisation already has under contract, and add only what is missing. The main models and platforms we work with are listed on the Services page.
- Prioritise existing contracted environmentsFirst consider what can run in already-contracted Microsoft, Google, or AWS environments, and add only what is missing.
- Data handling conditionsCompare services on whether inputs are used for training, storage location, retention period, and log handling.
- Japanese-language and long-text performanceBuild an evaluation set from real task data and compare accuracy and coverage across models.
- Cost and response timeFilter by cost per item processed and the latency acceptable for the task.
- On-premises executionFor tasks involving information that cannot leave the organisation, consider running an open model on internal infrastructure.
Roles of each tool
Responsible for decisions and generation
- OpenAI (GPT)General-purpose dialogue and generation
- ClaudeLong-form comprehension, agent control
- GeminiIntegration with Google Workspace
- Hugging FaceOpen models. On-premises inference
Running securely at enterprise scale
- Azure OpenAI ServiceUse in Microsoft environments; data handling conditions
- Amazon BedrockMulti-model access in AWS environments
- Microsoft 365 CopilotIntegration within Office and Teams
Structured as a business process
- DifyConversational apps and business flow construction
- n8nSystem integration and automation
- LangChainAgent and search layer implementation
Entry and exit points on the ground
- Notion AIDocument and task management
- Slack / TeamsInternal entry point and notifications
- Email / LINECustomer touch points
6 / SECURITY AND GOVERNANCE
Security and governance
Define what information may be passed to AI in advance, verify the data handling conditions of each service you use, and document it as an internal policy. Assurance is provided through both technical configuration and internal operations.
Define scope before sending
- Classifying informationSeparate personal data, confidential business data, and internal-only information. Define what may be passed to AI as an internal policy.
- Input pre-processingFields that do not need to be sent are removed or replaced before sending.
Verify for each service
- Use in trainingConfirm that inputs are not used for model training (opt-out settings or contract terms).
- Storage location and retentionVerify the storage location (country / region), retention period, and deletion method for data.
Define usage and review regularly
- Internal policyDocument what information may be handled, which services may be used, and the review process. Share through training.
- Copyright checksFor tasks generating documents or images, verify the treatment of referenced material and output in line with the Agency for Cultural Affairs guidance.
- Periodic reviewUpdate policies and settings in line with changes to service terms, laws, and guidelines.
7 / GUIDELINES
Alignment with public guidelines
The design principles explained in the sections above are mapped to the shared principles of the AI Guidelines for Business (Ministry of Internal Affairs and Communications / Ministry of Economy, Trade and Industry). The right-hand column shows the primary corresponding principle and the relevant section of this page.
Choose only the complexity required
Use workflows for fixed-process tasks and agents for tasks where decisions vary.
Related principleSafety / Innovation
See sectionDesigning RAG and AI agents
Make answers traceable to their source
Cite sources and return cases to humans when evidence is insufficient.
Related principleTransparency / Accountability
See sectionDesigning RAG and AI agents
Measure quality continuously
Use an evaluation set representative of real work and apply the same standard after every change.
Related principleSafety / Fairness
See sectionEvaluation and quality control
Build systems people can control
Embed permissions, approval, stopping, handover, and logging into operations.
Related principleHuman-centricity / Accountability
See sectionPermissions, logs and human approval
Define permitted information in advance
Confirm training-use policies and storage locations. Set the scope of acceptable information as an internal policy.
Related principlePrivacy protection / Security
See sectionSecurity and governance
Develop the people using it
Integrate training and retrospectives so staff can make corrections and drive improvements themselves.
Related principleEducation and literacy
See sectionServices: 03 Learning & Adoption
Public documents we refer to
- AI Guidelines for Business ver. 1.2 (Japanese) ↗Ministry of Internal Affairs and Communications / Ministry of Economy, Trade and Industry, March 2026
10 shared principles. The “Related principle” column in the table above uses principle names from this document.
- Guidelines for Introducing and Operating Text-Generative AI (Japanese) ↗Information-technology Promotion Agency (IPA), July 2024
Considerations by role for introduction, operation, and security. Reflected in the operation design and security sections.
- Alert on the use of generative AI services (Japanese) ↗Personal Information Protection Commission, June 2023
Confirming training-use of input data and use beyond original purpose. Basis for Principle 05.
- AI and Copyright: checklist and guidance (Japanese) ↗Agency for Cultural Affairs, March / July 2024
Checklist for tasks generating documents or images. Reflected in the copyright item of internal policy.
- Digital Skill Standard ver. 2.0 (Japanese) ↗Ministry of Economy, Trade and Industry / IPA, April 2026
Skill definitions for “AI implementation and operation” and “AI governance”. Mapped to training programme design.
The representative’s research background and the approach to applying research methods in implementation are on the About page.
Representative profile ↗