Why Does Codex Get Slower and Burn Through Usage Limits?

1. Why I Started Taking Context Management Seriously

I have been using large language models to solve problems for quite some time, particularly in programming and software development.

Like many people, I initially developed a habit of continuously asking questions, adding information, and refining requirements within the same session.

My assumption was simple: the more information an AI had about my project and previous discussions, the better it would understand what I wanted.

However, as conversations became longer, I began noticing several problems.

Even when my latest question had nothing to do with previous discussions, the AI would sometimes incorporate irrelevant historical information into its response.

Some answers appeared comprehensive but failed to address the actual problem in sufficient depth. Others seemed overly agreeable, following my assumptions rather than critically evaluating them.

In some cases, the AI gradually drifted away from my original requirements, even though I had clearly explained what I wanted.

I encountered similar problems while developing an enterprise knowledge base system using RAG (Retrieval-Augmented Generation) and large language model APIs.

At the time, I was concerned that the model might miss important information.

Therefore, I tried to provide as much retrieved knowledge and conversation history as possible.

I assumed that more information would help the model produce more accurate answers.

But the results were not always what I expected.

For example, when users asked about the detailed specifications of a particular electronic component, the model sometimes struggled to locate the exact information. Users had to ask several follow-up questions before receiving the answer they needed.

When users asked the model to calculate the total cost of all components in a device's Bill of Materials (BOM), it sometimes returned incorrect results.

Later, I experimented with improving retrieval accuracy, providing only information directly relevant to the current question, and adding clearer instructions about how the retrieved information should be processed.

For information retrieval and comprehension tasks, I found that this approach often produced more accurate and detailed responses.

Of course, BOM cost calculation involves another important issue:

Deterministic calculations should not rely entirely on a large language model.

For these tasks, a more reliable approach is to use software to retrieve and calculate the relevant data, then let the model explain the results.

These experiences gradually led me to an important realization:

Just because an LLM can process a large amount of context does not mean we should provide as much information as possible for every task.

Since then, I have deliberately managed the information I provide to AI models.

Over time, I extended this approach to ChatGPT, Codex, and the development of our own AI products.

2. My Understanding of Context Engineering: Not Less Context, but More Relevant Context

To better understand these observations, I studied several open-source resources explaining Transformer architectures and the internal mechanisms of large language models.

Two resources I found useful were:

• yeasy/llm_internals
• amitshekhariitbhu/transformers-explained

These materials helped me develop a better understanding of attention mechanisms, context processing, and related engineering concepts.

Combined with my practical experience, they helped shape my approach to context management.

2.1 More Context Does Not Necessarily Mean More Useful Information

Suppose we want Codex to modify the login functionality of an existing application.

We could provide the entire project history, architecture documentation, source code, dozens of previous development conversations, and design documents for unrelated modules.

The advantage is that the model potentially receives more information about the project.

However, this approach introduces several potential problems.

First, historical information may contain outdated requirements or abandoned implementation strategies.

Second, irrelevant information may make it more difficult for the model to identify what actually matters for the current task.

Third, longer contexts can increase processing costs and introduce more opportunities for conflicting instructions.

However, this does not mean shorter context is always better.

If we remove essential business rules, technical constraints, or interface definitions simply to reduce token usage, we may introduce new problems.

Therefore, my objective is:

Maximize the proportion of relevant and accurate information while preserving everything necessary to complete the task.

I think of this as managing the signal-to-noise ratio of context.

2.2 Context Window, KV Cache, and Markdown Files Are Different Things

It is important to distinguish three concepts.

The Context Window is the amount of context a model can process within its supported limits.

The KV Cache is a mechanism used during Transformer inference to cache the Key and Value representations of previously processed tokens.

The Markdown files I maintain in my projects, on the other hand, are part of an application-level context management system.

By selecting relevant files, compressing information, and dividing work into manageable tasks, we can reduce unnecessary input.

However, this does not mean we are directly managing the model's internal KV Cache.

Likewise, reducing context length does not automatically make a model reason more deeply.

What I actually want to achieve is much more practical:

Make it easier for the model to access complete, accurate, relevant, and internally consistent information when executing a task.

This principle became the foundation of my subsequent workflow.

3. My First Practical Approach: Turning AI Results into Reusable Markdown Files

Before I started using Codex, I primarily used ChatGPT and similar conversational AI tools for technical research, project analysis, and documentation.

Over time, I developed a particular habit.

After completing a stage of discussion, instead of continuously adding new tasks to the same conversation, I would summarize the confirmed results and save them in separate Markdown files.

For example, when discussing the technical architecture of a Spring Boot project, I might have several rounds of conversations with an AI.

We might compare different technologies, discuss their advantages and disadvantages, evaluate dependencies, and eventually select a particular solution.

Once the decision was made, I would preserve only the final technical choices, essential design rules, and necessary considerations.

Intermediate discussions and solutions that were ultimately rejected would generally not be included in the context files used for future tasks.

I also avoid repeatedly explaining common technical concepts that the model already understands.

For example, there is no need to explain what a Controller does in Spring Boot every time I start a new task.

What actually needs to be preserved is the architecture adopted by the current project, the external libraries it depends on, its specific requirements, and the rules that must be followed.

3.1 How I Organize These Files

Initially, I organized information by project, domain, and purpose.

For example:

project/
│
├── background/
│   ├── project-overview.md
│   └── business-context.md
│
├── workflows/
│   ├── development-process.md
│   └── incident-review-process.md
│
├── statistics/
│   ├── calculation-rules.md
│   └── reporting-rules.md
│
├── results/
│   ├── analysis-result.md
│   └── report.md
│
└── prompts/
    └── common-instructions.md

This is a simplified example. Different projects may require different directory structures.

I follow two important principles.

First, organize information according to its domain.

Closely related information should be kept together, while information belonging to different domains should be separated.

Second, distinguish background information, working rules, and final results.

For example, a project analysis report may be useful as a reference for future tasks.

However, it should not automatically become a permanent set of rules governing the entire project.

This organization allows me to provide only the files necessary for a particular task.

3.2 Why I Use Git to Manage These Files

These files are not documents that we write once and never touch again.

As a project progresses, we may discover new business requirements, modify previous designs, or identify errors in existing documentation.

I often ask AI to help update these files, then use Git to inspect the changes.

For example:

• Were new rules introduced?
• Was any previously valid information accidentally overwritten?
• Are abandoned solutions still present?
• Are there duplicated descriptions or conflicting requirements?

After reviewing and confirming the changes, I commit them to Git.

Over time, these documents evolve from ordinary conversation summaries into a maintainable project context system that can be selectively provided to AI.

When I started using Codex, I kept the same practice.

The difference is that Codex can now directly read and modify these files, so I no longer need to manually copy everything into each conversation.

4. What Really Changed My Codex Workflow: Managing Context Before Writing Code

As I gained more experience, I realized that simplifying documents before each AI interaction was not enough.

A more interesting question emerged:

Could I reduce the amount of context required for future development tasks by designing the software architecture differently from the beginning?

If a system contains highly coupled business logic, modifying one feature may require understanding a large amount of unrelated code.

In such a system, even a carefully written prompt may not be enough to keep an AI focused on a local change.

By contrast, when a system has a clear architecture, well-defined domain responsibilities, and stable public interfaces, it becomes much easier to establish the boundaries of an individual task.

This is why I believe software architecture remains extremely important in the age of AI-assisted programming.

In my practical development experience, its importance has become even more apparent.

4.1 How I Organize the Architecture

When developing The Inward Pioneer, our AI desktop application, I adopted an approach combining layered architecture with domain-based organization.

First, I use layered architecture to define technical responsibilities and dependency directions.

Then, within the business model layer, I organize functionality into domains based on actual business capabilities.

I further divide these domains into two categories.

Functional Domains: Provide relatively independent business capabilities, such as authentication and token management, conversation management, skill management, and music management.

Business Workflow Orchestration Domains: Coordinate multiple functional domains to complete business workflows and expose the necessary capabilities to the UI layer.

"Business Workflow Orchestration Domain" is a term I use within our project.

From the perspective of traditional layered architecture, some of its responsibilities are similar to business orchestration within an application service layer.

For example, a complete conversation workflow might need to:

1. Check the user's authentication status.
2. Invoke the conversation domain.
3. Use capabilities provided by the skill domain when necessary.
4. Return the result to the UI.

I do not want this logic duplicated across multiple UI components.

Instead, a workflow orchestration domain coordinates the capabilities provided by different functional domains.

I also define clear rules:

Different domains must communicate through their public interfaces.

A domain must not directly access another domain's internal implementation.

Circular dependencies between domains are prohibited.

This allows functional domains to behave like relatively independent components that can be combined to support different business workflows.

4.2 Why This Architecture Helps with Context Management

Suppose we need to modify the login functionality.

With a clearly defined architecture, we can quickly determine:

Which domain owns the current task?

Which models and interfaces need to be modified?

Which lower-level capabilities does the domain depend on?

Will the change affect other domains?

We can then provide Codex with the relevant architecture rules, source code, and functional requirements.

There is no need to make it reconstruct the entire project's business relationships for every local change.

More importantly, good modular design can reduce the impact of many everyday modifications.

This makes context easier to organize and reduces the risk of accidentally affecting unrelated functionality.

Of course, some complex tasks still require changes across multiple domains.

But with clear architectural boundaries, we can at least identify where those changes should occur and how they should be validated.

5. My Actual Development Workflow: Building The Inward Pioneer with ChatGPT and Codex

The following is an overview of the workflow I used while developing the macOS and Windows versions of The Inward Pioneer.

This is not a rigid process that every project must follow.

It is a working method I gradually developed based on the characteristics of our application.

Step 1: Use ChatGPT to Resolve General Technical Questions

Before writing project code, I use ChatGPT or other large language models to discuss the application's technology stack, programming languages, frameworks, dependencies, and specific technical questions.

I generally do not start with Codex at this stage.

My objective is to complete technical research and select an appropriate solution rather than immediately begin implementing code.

Once most of my questions have been resolved, I ask the AI to produce a concise technical summary.

The summary contains only the selected technologies, dependencies, essential constraints, and necessary considerations.

This becomes the project's technology stack document.

Step 2: Lead the Architecture Design and Establish Boundaries for AI

Next, I write the architecture design document.

It defines the layered architecture, responsibilities of each layer, domain boundaries, dependency relationships, public interface design principles, and testing and build requirements.

I spend a considerable amount of time on this stage.

I do not want to redesign the entire system every time a new feature is implemented.

However, I also avoid specifying every implementation detail in advance.

The architecture document only needs to establish the overall structure, responsibilities, boundaries, and critical rules.

Within those constraints, AI can make implementation decisions based on the existing code and specific requirements.

Step 3: Define Product Features and High-Level Business Workflows

After completing the architecture design, I prepare the product documentation.

This includes the application's primary features, the problems those features are intended to solve, and their high-level business workflows.

At this stage, I deliberately avoid specifying every UI detail and interaction.

In my experience, when product objectives and architectural boundaries are sufficiently clear, AI can sometimes propose interaction patterns and interface designs that I had not previously considered.

We can allow AI to generate solutions within the defined constraints, then refine them based on actual experience.

Step 4: Use Codex to Build a Runnable Application Framework

After preparing the technology stack, architecture, and product overview documents, I use Codex with a high reasoning effort setting to initialize the project and build its overall framework.

At this stage, I ask Codex to:

Create the project structure, necessary classes and interfaces, a complete end-to-end Demo workflow, test cases, and build and execution scripts according to the existing documentation.

For business functionality that has not yet been implemented, Stub implementations can temporarily return test data.

However, the application's basic UI and primary business workflows must be runnable.

This allows me to see a functioning application framework before beginning detailed feature development.

It also gives me an opportunity to verify whether the important architectural decisions meet my expectations.

For minor issues, I generally use a lighter model to make local corrections.

If I discover an architectural problem, I resolve it before proceeding with extensive feature development.

Step 5: Let AI Design the UI, Then Refine It

Once the initial application framework is running, I prepare a UI design language document.

This document establishes the overall visual style, design principles, and necessary constraints.

I then open a new session and ask Codex to improve the existing interface according to the product requirements and design language.

This stage usually involves several rounds of refinement based on actual results.

Once the design is sufficiently mature, I document the finalized color specifications, animation behaviors, and other reusable design rules.

The resulting UI document becomes context that can be reused in subsequent interface development tasks.

Step 6: Implement Features According to Domain and Task Boundaries

After completing the application framework, I generally create separate sessions for different development tasks.

A simple bug fix may require only a short instruction and a few relevant source files.

A complex feature may require additional product specifications, technical design documents, and relevant architecture documentation.

When a complete feature requires changes across multiple domains, I may use a higher-capability model to complete the work within a single session.

The key is not to enforce a rule that every session can modify only one domain.

Instead:

Determine which information, domains, and model capabilities are necessary based on the actual boundaries and complexity of the task.

6. A Practical Example: Implementing Authentication and Token Management with Codex

To demonstrate how this approach works in practice, let's use the authentication and token management functionality of The Inward Pioneer as an example.

This is a simplified educational example based on my actual development approach. The file paths and prompts are illustrative rather than verbatim records of historical conversations.

6.1 Start with an Explicit Domain Definition

In the architecture document, I define authentication and token management as a functional domain.

For example:

The architecture documentation also defines how functional domains and business workflow orchestration domains should collaborate.

Therefore, when Codex implements authentication, it does not need to determine where authentication logic belongs or redesign the application's overall architecture.

6.2 How We Organize Our Project Documentation

During development, I store the project's core context in a shared docs directory.

For example:

project/
│
├── docs/
│   ├── technical-stack.md
│   ├── architecture.md
│   ├── product.md
│   ├── ui-design.md
│   ├── team-collaboration-rules.md
│   │
│   ├── product-detailed-design/
│   │   ├── login.md
│   │   └── chat.md
│   │
│   └── technical-detailed-design/
│       ├── login.md
│       └── chat.md
│
├── src/
│   ├── storage/
│   ├── network/
│   ├── utils/
│   ├── model/
│   │   ├── authentication/
│   │   └── demo/
│   └── view/
│
└── tests/

Each document serves a specific purpose.

technical-stack.md

Records the project's technology stack, programming languages, external libraries, and relevant technical constraints.

architecture.md

Defines the overall architecture, layer responsibilities, domain boundaries, dependency relationships, and development conventions.

product.md

Records the application's primary features and high-level business workflows.

ui-design.md

Contains established interface design principles and visual specifications.

team-collaboration-rules.md

Records development and collaboration rules that all team members are expected to follow.

product-detailed-design/

Contains detailed descriptions of individual features, including behavior, constraints, error handling, and expected user interactions.

These documents are maintained by product team members.

technical-detailed-design/

Contains API invocation workflows, key interfaces, implementation constraints, and technical considerations for individual features.

These documents are maintained by technical team members.

For existing features, the technical design documents may also contain the locations of relevant source files, allowing Codex to identify the appropriate implementation quickly.

These documents do not need to be completed all at once.

Different projects may also use different structures.

However, I do not consider them merely traditional project documentation.

These files preserve the designs, rules, and decisions already established by the team. They form the long-term context through which human team members and AI collaborate.

Different team members can read, modify, review, and reuse them.

When someone opens a new AI session, they do not need to reconstruct every decision made in previous conversations.

6.3 Which Context Do I Provide for the Authentication Task?

Suppose the overall architecture is already complete and we are ready to implement authentication.

I select the necessary context according to the task.

Documents related to unrelated domains, such as music or meditation, do not need to be provided.

I also do not provide the entire history of previous development conversations.

If Codex discovers that additional code is required during implementation, it can continue reading the necessary files.

My principle is to let AI start with the most relevant information, not to prevent it from accessing additional information required to complete the task.

6.4 A Codex Prompt You Can Adapt

In my actual workflow, I sometimes paste relevant documents directly into the conversation.

At other times, I provide their full paths and let Codex read them.

For relatively complex tasks, I generally begin a new session by asking Codex to read the necessary documentation and confirm its understanding before implementation.

The following is a simplified educational example.

First instruction: Read and understand the task context.

Task: Implement authentication and token management.

1. Read the context

Read the following files first:

1. docs/architecture.md
2. docs/product-detailed-design/login.md
3. docs/technical-detailed-design/login.md
4. docs/ui-design.md
5. Existing code in src/model/authentication/
6. Existing code in src/model/demo/

After reading them, briefly summarize:

1. The architectural layers and domains involved
   in this task, including the key constraints
   and rules that must be followed.

2. The capabilities already implemented
   in the authentication domain.

3. The technical implementation of the Demo
   domain, including how it combines capabilities
   from other domains to implement business
   workflows.

4. The functionality that needs to be added
   or modified for the current task.

Do not repeat the entire project architecture.

Only summarize conclusions directly relevant
to the current task.

Do not modify any code at this stage.

After confirming that Codex has understood the relevant context, I ask it to execute the task.

Second instruction: Implement the functionality and validate the results.

2. Task objectives

Based on the existing project:

Follow the technical design defined in:

docs/technical-detailed-design/login.md

Implement the authentication and token
management functionality specified in:

docs/product-detailed-design/login.md


3. Architectural constraints

Strictly follow the architectural rules
and domain boundaries defined in:

docs/architecture.md


4. Implementation requirements

Prioritize reusing existing code, models,
technology stacks, and infrastructure.

Ask for my confirmation before introducing
new dependencies or upgrading existing
dependency versions.

Only modify code necessary to complete
the current task.

Do not redesign the entire project architecture
to implement this feature.


5. Validation requirements

Add the necessary test cases according
to the project's existing testing conventions.

Run the relevant tests to verify that
the functionality meets the requirements.

Execute the existing build scripts to confirm
that the changes do not introduce build errors.

Review the scope of the changes and confirm
that unrelated domains have not been modified.

Finally, briefly report:

1. Which features were implemented.
2. Which files were modified.
3. The test and build results.
4. Any unresolved issues.
5. Whether any related documentation
   needs to be updated.

If documentation changes are necessary,
ask for my confirmation before making them.

This does not mean I use such a detailed prompt for every task.

If I only need to fix a simple bug, I may provide the paths to the relevant source files and briefly describe the problem.

I also do not require Codex to read the architecture document for every task.

Context should be selected according to the requirements of the task, rather than treating every rule and document as mandatory input.

This is one reason I generally do not put all project documentation into AGENTS.md and require it to be loaded for every task.

Instead, I prefer explicitly specifying which files Codex should read when they are needed.

6.5 How Do I Validate the Results?

During project initialization, I already require Codex to establish testing conventions, build scripts, and execution scripts.

Therefore, after completing a task, I can use these existing tools to validate the result.

For authentication, I check successful login, failed login, logout, authentication state changes, and token management behavior.

I also run the relevant unit and integration tests, then execute the build scripts.

For features that can be verified through the UI, I launch the application and manually test the actual behavior.

Additionally, I use Git Diff to review the modified files and code.

This helps me identify unintended changes to unrelated domains.

One important point:

A successful build does not guarantee correct business behavior, and passing automated tests does not eliminate the need for human review.

Automated testing, actual execution, and code review should complement one another.

6.6 How Do I Update the Context After Completing the Task?

Suppose authentication has been implemented according to the established architecture, without introducing new public interfaces or architectural rules.

In that case, I generally do not need to rewrite the architecture document.

However, if the implementation introduces new domain responsibilities, public interfaces, or constraints that should apply to future tasks, the relevant documentation must be updated.

Similarly, if the product behavior or workflow changes, I update the corresponding product documentation.

For architecture documents, I preserve only the finalized designs, responsibilities, boundaries, and constraints.

I do not include every code implementation detail or retain intermediate solutions that were ultimately rejected.

This means that when a future task requires authentication functionality, Codex can quickly understand the current design by reading the relevant documentation.

7. Context Management Is Not Just for Small Tasks: A Real Cross-Domain Refactoring Experience

The authentication example demonstrates how I approach everyday feature development.

However, real software projects inevitably require changes across multiple modules and sometimes even adjustments to the original architecture.

For these tasks, I do not mechanically insist that every domain must be modified in a separate session.

When multiple domains have clear dependencies and the objective is to complete a coordinated architectural change, I may use a higher-capability model to complete the refactoring within a single session.

The important part is to define the scope, steps, and constraints before execution.

7.1 A Problem We Encountered During Development

The Inward Pioneer originally supported multiple characters inspired by different schools of Eastern philosophy.

We used a Plugin mechanism to organize the functionality associated with different characters.

At the time, we believed that each character's skills might influence its conversational behavior.

Therefore, we placed some shared business logic inside individual Plugins, including general conversation handling and audio sending and receiving.

Later, we discovered that this approach increased maintenance costs.

Whenever the conversation workflow needed to change, we often had to make similar modifications across multiple Plugins.

We eventually decided to reorganize the domain responsibilities.

The shared conversation and audio-processing logic would be moved into a common conversation domain.

Individual Plugins would retain the responsibilities specific to their respective characters.

7.2 How I Used Codex to Complete the Refactoring

This type of task is fundamentally different from fixing a simple bug.

I prepare a relatively detailed Markdown document describing the refactoring objectives and constraints.

For example:

Which shared logic must be removed from the Plugins?

Which domain should own that logic after refactoring?

Which public interfaces must remain stable?

How should the affected domains communicate?

Which unrelated domains must not be modified?

I do not necessarily specify every internal implementation detail.

Codex can inspect the existing code to understand how the current implementation works.

Instead, I focus on clearly describing the refactoring objectives, changes in domain responsibilities, dependency relationships, and boundaries that must not be violated.

I then ask Codex to complete the refactoring based on the document and relevant source code.

The results must pass the existing tests.

Once the implementation meets my expectations, I ask Codex to update the architecture documentation accordingly.

7.3 What This Experience Taught Me

Context Engineering does not mean every task must be small.

Nor does it mean every task should use a lightweight model.

For complex tasks that genuinely require changes across multiple domains, we need to provide sufficiently complete context.

However, this still does not mean loading the entire project's documentation and source code.

What we actually need is:

The domains involved in the current refactoring, relevant source code, existing dependencies, the target architecture, modification boundaries, and validation requirements.

This allows us to support complex development tasks while avoiding unnecessary information.

8. When Codex Repeatedly Fails to Fix a Problem, I Rebuild the Context Instead of Extending the Conversation Indefinitely

Even with carefully prepared context, AI can still make mistakes.

For example, Codex may modify code that subsequently fails to compile, or the application may behave differently from what I expected.

When an issue first appears, I generally ask Codex to fix it within the current session.

However, if several consecutive attempts fail to resolve the problem, or the conversation has already undergone context compaction, I consider starting a new session.

I do not simply copy the entire previous conversation into the new session.

Instead, I create a new Markdown document describing the current problem.

It generally includes:

# Current Issue

## 1. Observed Problem

Describe what actually happened.

## 2. Expected Behavior

Describe what the correct behavior should be.

## 3. Steps to Reproduce

List the steps required to reproduce the issue.

## 4. Latest Error Information

Include the actual error message, relevant logs,
or failed test results.

## 5. Previously Attempted Solutions

Briefly summarize the modifications that have
already been attempted without resolving the issue.

## 6. Possible Causes

Record the suspected causes.

Clearly identify these as hypotheses that
still need to be verified.

## 7. Relevant Files

List the source files or directories related
to the current issue.

I then open a new session and ask Codex to analyze the problem using this information.

For more complex issues, I may select a model with higher reasoning capabilities.

When it is safe to do so, I may also use Git to return the codebase to a previously confirmed stable state before attempting another solution.

Previously attempted solutions are not entirely useless.

They can help AI avoid repeating approaches that have already failed.

However, I preserve only the necessary information about those attempts rather than copying every analysis, speculation, and modification from the old conversation.

My objective is to preserve verified facts while preventing unconfirmed assumptions from previous attempts from dominating the new analysis.

9. Can This Approach Reduce Codex Usage?

In my actual development work, I have observed several practical benefits.

When the architecture, domain responsibilities, and context documentation are clear, many everyday development tasks and bug fixes do not require the highest available reasoning effort.

I can choose different models according to the complexity of the task.

For relatively simple tasks, I use a lighter model.

For architectural design, large-scale refactoring, or complex problem analysis, I may use a more capable model.

Because individual development tasks have clearly defined boundaries, I also spend less time repeatedly explaining the entire project to AI.

While developing The Inward Pioneer, I found that most everyday tasks could be completed using this approach.

My available Codex usage has generally been sufficient to support my development workflow.

However, there is an important qualification.

These are observations from my own development experience, not conclusions from a controlled experiment.

Codex usage depends on several factors, including model selection, task complexity, execution processes, and tool usage.

Reducing context does not guarantee lower usage for every task.

For example, a new session may need to reread source files and rebuild its understanding of the project.

Similarly, if a task genuinely requires complex reasoning or extensive modifications, shortening the prompt alone may not substantially reduce its execution cost.

Therefore, I would not summarize my approach as "less context always means lower usage."

Instead, I think of it this way:

Good software architecture and context organization allow more tasks to be completed within clear boundaries, making it easier to select an appropriate model for each task.

10. This Approach Also Applies to Non-Developers

Although I have used software development examples throughout this article, the underlying approach is not limited to programming.

Consider a team using ChatGPT for product planning, market analysis, or operations.

The team can organize its information into separate categories:

Project background, product rules, business workflows, data calculation methods, raw data, historical analysis results, and reusable instructions.

For example, data calculation rules should be maintained consistently across the team.

Otherwise, different team members may use different definitions and calculation methods in separate AI conversations.

When product requirements change, the relevant product documentation should be updated rather than allowing everyone to continue relying on outdated discussion history.

When analyzing operational data, we should provide only the data, calculation rules, and background necessary for the current analysis.

There is no need to include the entire product history, unrelated business data, or every previous analysis report.

In this way, concepts from software architecture—layering, modularity, high cohesion, and low coupling—can also be applied to everyday knowledge work.

A "domain" does not necessarily need to represent a software module.

It can also represent an independent business process, responsibility, or knowledge area.

The essential idea is not to follow a particular directory structure, but to give different types of information clear responsibilities and boundaries so they can be maintained independently and combined when necessary.

Existing large projects do not necessarily need to be rebuilt from scratch.

A team can start with its most frequently used business domains and work with AI to organize the relevant background information, working rules, and historical decisions.

Over time, this creates a reusable context system.

11. Final Thoughts: In the Age of AI Coding, Architecture and Human Judgment Matter More Than Ever

Looking back at my experience using ChatGPT and Codex, I believe the most valuable lesson is not a particular prompt.

It is a set of working habits that I have gradually developed.

First, divide projects into clearly defined layers and domains.

Keep related information together and establish explicit interfaces between different domains.

Second, organize confirmed background information, rules, designs, and results into concise, accurate, and maintainable documents.

When executing a task, select the context according to actual requirements instead of providing every available document.

Finally, invest sufficient effort in architecture design and establishing essential rules before beginning extensive development.

Validate AI-generated results through testing, code review, and actual execution.

Through my work, I have become increasingly convinced of one thing:

AI can help us write code faster, but humans still need to decide how a project should be structured, which information deserves to be preserved, and whether the final result actually meets our requirements.

Context Engineering is not merely a technique for reducing token consumption.

It involves information organization, knowledge management, software architecture, task planning, and result validation.

As AI tools develop stronger long-context capabilities, automatic context compaction, and autonomous execution mechanisms, they will become increasingly capable of handling complex tasks.

However, I still believe that actively managing a project's critical knowledge and decisions has value beyond the capabilities of any individual AI model.

That information serves not only the model we use today, but also our entire team and whatever AI tools we may use in the future.

What I want to build is not a project that can only continue functioning because a particular AI session remains open.

I want a project that can continue moving forward even after the current session is closed—whether the next task is handled by another developer, another AI model, or another team.

This is the approach I have been developing and applying while building The Inward Pioneer and other products at AI Pioneer.

About AI Pioneer and The Inward Pioneer

AI Pioneer is an AI technology company focused on bringing Eastern wisdom into everyday life through artificial intelligence.

Our product, The Inward Pioneer, is a Taoist-inspired AI desktop companion available for macOS and Windows.

Rather than attempting to become a general-purpose AI assistant that answers every question, it is designed to offer different perspectives when people face challenges in their work, relationships, and everyday lives.

Through thoughtful conversations, it helps users gain clarity about their situations and develop their own judgment.

The Context Engineering and AI-assisted development methods shared in this article are drawn from our practical experience building real products.

Official website: https://www.smartpioneer.com.cn/