Claude Code was born when Boris gave an AI free run of his terminal. OpenClaw was born when Peter gave an AI unrestricted control of his computer. Both came from access, not from a cleverer model.

And it’s the exact thing we’re trained never to do.

As engineers, guarding secrets is drilled into us. Least privilege, rotate the keys, never hand over the whole machine. For good reasons. The model itself has absorbed the same instinct: throw a real credential at it and it gets visibly nervous and starts telling you to rotate it.

Now the uncomfortable part. If Boris and Peter had started by designing the permission model, they’d still be designing the permission model. The breakthrough needed someone willing to hand over the keys and watch what happened.

So if you actually want to see what an agent can do, stop shielding it from your own systems. Forget the training. Trust the model. Give it real access — your repos, your data, your tools — now, not after you’ve built the perfect sandbox.

This is what the access buys you. Without it, the agent is all brain and no hands. It figures out exactly what needs to happen and then leaves the doing to you. So you become its hands and legs, executing the decisions of a brain that isn’t yours. Give it the keys and it gets its own hands and legs. It opens the PR and merges it, sends the email, runs the server. The distance between deciding something and it being done, the distance that used to be you, mostly disappears. You get your head back for the part that was always yours: choosing what’s worth doing, and judging whether it came back right.

And no, I’m not saying paste your production secrets into a chat box and walk away. Give the agent a machine of its own. Hetzner will rent you a real physical server for less than half the price of an equivalent cloud VPS. A whole machine, not a thin slice of someone else’s. On a box like that the agent has room to actually work: spin up dozens of worktrees that mirror production, run hundreds of processes at once, drive a handful of browsers in parallel. No tiptoeing around a cramped VPS.

Never set up a server in your life? You don’t need to learn now. Hand the agent the SSH key, or just the root password, and it stands the whole thing up for you, which is the same move this whole post is about. It stays on. If it ever goes sideways, you wipe the box and start fresh. That’s your kill switch.

The ones who pull ahead over the next two years won’t be the best prompters. They’ll be the ones who gave their agent real access and let it do the work.

In a small company, the CTO often becomes the point where business intent gets translated into technical reality.

The visible work is architecture, infrastructure, code, pull requests, and technical judgment. The less visible work is holding the map of software, data, product history, customer context, operations, old edge cases, and hidden constraints.

Someone needs a report, and the CTO knows where the data lives. Someone asks whether a new idea is simple or expensive, and the CTO translates the question into schema, permissions, code paths, deployment risk, and sequencing. Someone sees something odd in the business, and the CTO becomes the query engine, the debugger, and the explainer.

That is useful for a while. Early on, it gives the company speed. But as the company grows, it turns technical leadership into a reactive translation queue, and the CTO becomes a bottleneck precisely because they have too much context.

The internal AI work I have been doing at Hero Group is my attempt to move the company toward that kind of AI-native operating model.

One part of that work, and the part I want to focus on here, is the internal operating layer: how our own team gets answers, creates artifacts, follows up on decisions, inspects systems, and turns intent into work without routing every step through me. The goal is not to remove engineering judgment. The goal is to reserve engineering judgment for work that actually needs it, instead of spending it on translation work created by systems that are hard for non-engineers to operate directly.

That is the lens through which I watched YC’s recent discussion on becoming an AI-native company. What stayed with me was how familiar the operational problem felt. Their examples were different, but the shape was similar: give teams a way to express intent in natural language, connect the agent to real company context, and build the review boundaries around the places where it can act.

AI becomes genuinely useful inside a company when it is connected to the company’s context, tools, permissions, workflows, review loops, and artifacts. Without that, it is mostly a clever assistant sitting outside the real work.

A four-person team, one internal AI agent, and three months of real usage.

AI-Native Is An Operating Model

The phrase “AI-native company” is easy to flatten into marketing. It can mean a product has AI features, employees are encouraged to use ChatGPT, or a company has a few internal automations running in the background. The version I care about is operational: a company where internal systems are redesigned around the assumption that AI can safely sit close to the work, read context, use tools, create first drafts, inspect systems, ask for review, produce artifacts, and hand work back to humans at the right boundary.

That requires more than prompts. It requires architecture.

It also requires meeting people where they already are.

For us, that means Hero Hero, our internal AI agent, lives directly in the tools the team already uses: Slack for everyday interaction and Basecamp for the more durable project-management layer. Our team was already comfortable in those places. If the AI required everyone to move into a new cockpit, it would have become one more system to remember. Putting the agent inside the surfaces where work already happens made it feel less like “using an AI tool” and more like asking for help in the normal flow of work.

The system still has to know where context lives, which tools exist, what it can do without approval, what it must escalate, how to produce outputs people can inspect, and how to leave a trail. But the interface has to stay natural. AI should adapt to the team more than the team adapts to the AI.

Hero Hero Is An Operating Interface

Chat is the surface of an internal AI system, not the substance.

Hero Hero, or HH internally, matters because it sits near the actual operating context of the company. It can work with internal documentation, plans, code, read-only data, Slack, Basecamp, email, browser sessions, Drive, Calendar, call transcripts, and generated artifacts. More importantly, it has instructions and boundaries around how to use that context: when to inspect, when to summarize, when to draft, when to create a pull request, when to ask another AI for review, when to ask a human, and when to refuse.

That changes the shape of everyday work.

The primary interface is human language. A teammate should be able to explain what they want in the words they naturally use, not in schema names, product internals, or ticketing-system language. They should be able to send a message, screenshot, voice note, video, transcript, or rough idea and have the system understand enough to move the work forward. They should also be able to get work back in useful formats: a report, task, draft email, walkthrough, product note, implementation plan, generated asset, or pull request.

A teammate asks a business question in plain English, and Hero Hero fetches the answer.

Not every useful AI interaction has to look formal.

Hero Hero is useful for creative work too, not only technical or analytical work.

The important shift is not that AI answers questions. The important shift is that AI moves intent closer to the systems and artifacts where real work happens.

Where The Leverage Shows Up

One of the strongest internal patterns has been read-only access to production context.

Companies often discuss AI access as if there are only two choices: keep the model away from real systems, or give it dangerous power too early. The useful middle is controlled read-only visibility. Let the agent inspect real context, but make mutation narrow, explicit, auditable, and human-reviewed.

Read-only context changes the quality of answers immediately. A model that cannot see the business gives generic advice. A model that can safely inspect the actual state of the business can answer grounded questions, find inconsistencies, explain trends, and help people understand what is happening without waiting for the CTO to manually run the first query.

The bigger change is that people ask more questions. When asking a data question requires finding the right person, explaining the context, waiting for the query, and then asking the follow-up, many questions never get asked. When the interface is natural language and the system can fetch the context safely, curiosity becomes cheaper. We ask about user behavior, product interactions, documents, operations, and internal workflows because the cost of asking has collapsed.

That does not remove the need for boundaries. Some sensitive data should remain encrypted so even the agent cannot read it. That is a feature, not a limitation. A serious internal AI system should create more visibility where visibility is useful and hard walls where hard walls are necessary.

Another leverage point is the tool and skill layer. In the YC discussion of tool registries, the important idea is not the packaging. The important idea is resolution: can the agent discover what it is capable of doing, understand the right entry point, and use the right tool without a human manually wiring every step?

That registry can be code. It can be markdown. It can be YAML. It can be a set of internal skills, CLIs, scripts, browser tools, and documents. As models get better, a well-written markdown file can be a perfectly serious part of the architecture. The question is whether the system is legible enough for the agent to resolve capabilities correctly and safe enough for humans to trust the result.

From review to merge to deployment, internal AI can move work across real engineering steps.

Model routing is part of the same layer. Hero Hero knows that different models have different strengths and weaknesses. Some are better at technical architecture, some at design, some at image generation, some at transcription, some at long-form reasoning, some at fast routine work, and some as reviewers. A good internal system should not pretend that one model is always the answer. It should know when to route, when to review, when to ask another AI for a second opinion, and when to stop for a human.

Company context can matter more than a standalone AI tool.

Memory and scheduling matter too. Some work is not a one-shot answer. The agent needs to remember durable context, follow up at the right time, and connect a conversation today to a task or artifact tomorrow. That is when AI starts feeling less like a search box and more like operating infrastructure.

What Changed In The Team

The best part of this work has been watching the team use it.

People no longer wait for me to answer every product, data, or implementation question. They ask Hero Hero. They ask in the language they already use. Sometimes they give it a screenshot. Sometimes they give it a voice note. Sometimes they ask for a plan, a draft, a report, a product idea, a launch outline, a social post, or an analysis. Everyone seems to address it like a real teammate, which says something important about where the interface is working.

The work a 65-person ops team was trained to do — read the ticket, look up the customer, troubleshoot, respond — now happens in natural language, in minutes.

The behavior change is simple: people stop searching manually and ask the internal agent.

Our COO, who has never been a frontend engineer, is now actively driving product UI and UX decisions. He talks to users, collects feedback, works through ideas, and can drive improvements with HH in the loop instead of waiting for every detail to be translated through engineering first. Other people are creating launch plans, strategy documents, social media drafts, product ideas, and internal analysis with a level of independence that would have been hard to create with traditional tools.

When the agent is in the workflow, people start treating it like a teammate.

One internal agent can create consistency across product, brand, and communication work.

This is the part that is easy to underestimate from the outside. The value is not only that work gets done faster. The value is that people feel more able to contribute. They do not come only with ideas anymore. More often, they come with a first version, a prototype, a draft, or a concrete artifact that can be reviewed.

That changes morale. It changes ownership. It changes how much of the company can move without waiting for one technical person to translate every next step.

Once the agent sits close to enough of the company’s work, the team stops calling it a tool and starts calling it core infrastructure.

Conversations Become Company Memory

A lot of important work does not begin in a dashboard or a ticketing system. It begins in conversation.

Slack messages, Basecamp threads, calls, transcripts, screenshots, rough notes, and half-formed ideas are all part of the company’s memory. Traditional software loses a lot of this because the conversation and the system of action are separate. Someone has to remember what was said, write the task, update the document, notify the right person, and follow up later.

An internal AI operating layer can reduce that loss if it is built with the right boundaries. This is not an argument for exposing every private conversation to every workflow. Scope matters. Consent matters. Context should be used for the purpose it was collected for, and sensitive material should be summarized carefully or not used at all.

Within the right boundary, conversations become operational inputs. A meeting can become a decision record. A transcript can become follow-up work. A screenshot can become design feedback. A rough conversation can become a draft, a report, a task, a pull request, or a set of next steps.

We already record transcripts, and HH can use that context. We are also building toward a workflow where HH can be addressed directly from a call and then act once the transcript becomes available. That may sound like a small interface change, but I think it points to a bigger shift: the next system of action will often begin inside the conversation that already happened.

The team noticed the system feeling like a different operating mode, not just another tool.

The Challenges Are Real

None of this works by magic.

You have to decide what the agent can see, what it cannot see, what it can do, what it must never do, and when a human has to review the output. You have to keep context fresh. You have to prevent tool sprawl. You have to keep improving prompts, skills, routing, and review paths as models change. You have to accept that the workflow you build this month may need to be improved next month, or next week.

That pace is uncomfortable, but it is also the nature of the work. AI-native operations are not a one-time implementation. They are a living operating layer.

The companies that get good at this will not be the ones that install one tool and call it done. They will be the ones that keep improving how context, tools, models, review, and artifacts fit together.

What This Means For The CTO Role

The CTO role does not disappear in this model. It becomes more architectural.

Instead of being the person who manually translates every operational question into a technical action, the CTO designs the system that lets more people interact with technical context safely. That means building boundaries, choosing tools, exposing the right context, making review paths explicit, and deciding where automation should stop.

This is the kind of technical leadership I think will matter more over the next few years.

Not adding AI as decoration. Not wrapping a model in a thin interface and calling it transformation. The harder layer is the internal operating system that lets a company move faster because AI has controlled access to context, tools, workflows, and review.

That still requires deep engineering. Rails matters. Databases matter. Infrastructure matters. Logs, queues, permissions, tests, deployment discipline, and product judgment all matter. AI does not make the underlying systems less important. It makes weak underlying systems more dangerous because the model can only reason from the context and tools you give it.

My own work changed before the company layer changed. I have always treated my workflow as infrastructure: editor habits, terminal setup, TDD loops, key bindings, tmux, small scripts, and tiny improvements that compound over years. AI changed the ingredients, but not the habit. The workflow now changes faster. Sometimes the improvement is a prompt. Sometimes it is an agent rule. Sometimes it is a new model route, a review step, a tool, or a better way to hand work between humans and AI.

Since moving into AI-native coding tools, I have stopped treating the traditional editor as my main interface. That personal shift made a larger question unavoidable: if AI can become the interface through which I build software, why can it not become the interface through which more of the company operates?

That is the question I have been answering inside Hero Group.

Why The YC Conversation Mattered

The YC conversation mattered because it gave better language to patterns I was already seeing from the inside.

The patterns were familiar: company context matters, agents need tools, read-only access changes what people are willing to ask, tool registries behave like resolvers, model routing and review matter, conversations and transcripts can become useful inputs, and outputs should often be artifacts rather than chat replies.

We are not finished. Some parts are already working well, some parts are rough, and some parts are clearly on the roadmap, including deeper automation and analysis around marketing and social distribution. But the direction feels right.

The larger shift is cultural as much as technical. In AI-native teams, people do not only bring ideas to the table. They increasingly bring drafts, prototypes, analyses, plans, and working artifacts. The distance between “I have a thought” and “here is something we can inspect” gets much shorter.

That is what I am seeing inside Hero Group.

For me, that is what AI-native means in practice. It is not a slogan and it is not just a better chatbot. It is a company gradually rebuilding its internal operating layer around context, tools, safety, artifacts, and human judgment.

That is the work I am doing.

If you are building in this direction, or trying to make AI useful inside a real company, I would be happy to compare notes.

Reach me at: tannakartikey@gmail.com

Originally published on LinkedIn.

By now, most CTOs have already seen AI change their personal output dramatically. Whether you call it 10x, 100x, or something else, the practical reality is the same: they can think, analyze, design, automate, review, and ship across a much wider surface area than before. That is not the hard part anymore. The harder question is whether that leverage stays with the CTO, or becomes part of how the company operates.

I came to that question through my own workflow. For years, I have been looking for ways to reduce the distance between an idea and a working result. Earlier, that meant better development workflows, internal tooling, automation, test loops, and small process improvements. Now it also means prompts, agents, permissions, review gates, and workflows where humans and AI can hand work back and forth safely.

But that is still only personal leverage.

If the CTO is the only person who knows how to use AI well, the company has not become AI-native. It has only made its existing technical bottleneck faster.

In a small company, many questions are not purely engineering questions, but they still route through engineering because the context lives there. Is this product idea simple or expensive? What does the data actually say? Why did a customer see this behavior? Can this feedback become a workflow, a report, a design direction, or a product change? Historically, the CTO translates those questions into database queries, code paths, product history, permissions, deployment risk, and sequencing.

The new CTO role is to reduce that translation cost without creating chaos.

At Hero Group, the internal AI work has taught me this directly. The goal was not to give everyone a chatbot. The goal was to put an agent near real company context and real working surfaces, with enough access to be useful and enough boundaries to be trusted. Teammates should be able to ask product, data, design, and workflow questions in natural language and get back something concrete enough to inspect: a data read, a product note, or a proposed change.

Within the right boundaries, it can go further. A teammate can review an AI-built change, approve it, and have it move through merge and deployment without waiting for the engineering team to translate every request into code and release steps.

The important signal is not only that engineers can ship faster. The stronger signal is when non-engineers can move work forward without waiting for every step to pass through engineering.

This is not theoretical for us. In one recent three-month period, about 84% of the questions our internal agent answered came from non-engineers. Our COO, who has never been a frontend engineer, now drives product UI and UX decisions with the agent in the loop: talking to users, working through ideas, and pushing improvements forward.

I wrote more about the operating layer behind that here: Removing the CTO as the Bottleneck.

But passing this leverage to the company is not only a tooling problem. It is a cultural transition.

People do not all work the same way. Some immediately see how to use AI. Some need examples close to their actual work. Some over-trust it; others under-use it. The CTO cannot just build agents and assume adoption will happen.

The internal AI layer is a product. Its users are the people inside the company. The CTO has to watch where they hesitate, which outputs save time, which ones create more review work, and which handoffs feel confusing. Then the system changes: prompts, tools, permissions, and review paths improve as the team learns.

The company learns how to work with AI through real usage, not a policy change. People learn by trying it in their own work, seeing how teammates use it, and building the judgment to ask better questions, inspect outputs, know when to trust and when to challenge, escalate risk, and recognize when a human decision is still the point.

At Hero Group, I have noticed that once teammates start using AI in the flow of real work, they do not really go back to the old default. When the models are unavailable, it feels less like one tool is down and more like the internet is down. The company can still work, but the normal rhythm has changed.

There is also a cognitive cost. AI can help someone produce far more than before, but that often means they now have far more to evaluate: more drafts, more options, more decisions, more context switching, and higher expectations. The work changes shape; it does not remove the need for human judgment.

This is why the CTO has to understand the team, not only the models. AI can be extremely capable and strangely wrong in the same workflow. Helping a company navigate that reality is now part of technical leadership.

Once that adoption loop starts, attention shifts to the work itself. In the age of AI, part of the job is to keep asking: what should no longer require a human to start from zero?

Reports, follow-ups, QA passes, customer summaries, and recurring analysis all contain repeatable patterns. Some parts still need judgment. Some parts need approval. Some parts should never be automated. But many parts can be prepared, checked, summarized, routed, or drafted by an agent before a person touches them.

That does not usually happen because every person in the company independently imagines the future workflow. Most people are focused on doing their job with the tools in front of them. They may not know what can be delegated to AI, what context the agent could safely use, or how a recurring process could be redesigned.

The CTO has to see those patterns and keep looking at ordinary company work, asking where AI can remove repetition, shorten feedback loops, and free people for the parts that actually need human judgment.

This changes what technical leadership has to design. The architecture is not only APIs, queues, Rails models, databases, and infrastructure. It is the AI boundary too: what context the agent can read, which tools it can use, where writes require approval, how actions are logged, what it must refuse, and where human judgment remains the gate.

Cost management becomes part of that system too. If agents are doing real work across the company, the CTO has to know which tasks need the strongest models, which can run on cheaper models, where caching or batching helps, and where another long agent loop is not worth the cost. AI spend should be visible enough to manage, but not controlled so tightly that people stop using it for high-leverage work.

The same redesign has to happen inside engineering itself. It is not only the CTO using AI. Engineers are using it too. Agents are writing code, tests, plans, migrations, and fixes. The amount of code a team can produce in a day is changing faster than the old review process can absorb.

If engineering output becomes 10x but review, testing, deployment, and production monitoring remain 1x, the bottleneck does not disappear. It moves into the pipeline, and the risk goes up with it.

The CTO cannot personally review every AI-assisted diff. A team lead cannot manually inspect every line generated across every branch. But saying no to the speed is not the answer either. The responsibility is to redesign the delivery system around the new output rate.

That means stronger automated tests, AI-assisted review, preview environments, staged rollouts, observability, rollback paths, and clear human gates for anything that can affect customers, money, data, or production stability.

The engineering pipeline has to become AI-native too.

Not because AI should approve its own work blindly, but because a human-only review process cannot be the only safety layer when the volume and speed of work have changed.

That is not a softer version of engineering. It is deeper engineering applied to the operating model of the company. It also requires more understanding of the business, not less. You cannot build useful AI leverage from the codebase alone. You need to understand where operations loses time, what customer-facing teams repeatedly ask for, how product decisions are made, where design gets stuck, and which mistakes would actually hurt customers, money, production, or trust.

This is why I do not think the CTO role becomes smaller with AI. It becomes more architectural. The CTO stops being only the person who makes technical decisions and becomes the person responsible for how technical context can be safely used across the company.

AI can multiply output. It can also multiply bad assumptions, weak permissions, unclear ownership, and messy context. The companies that get this right will not be the ones that simply encourage everyone to use AI tools. They will be the ones that deliberately build the operating layer around AI use.

That operating layer also has to be designed for where the models are going, not only where they are today.

Model capability is moving too quickly for a company to rebuild its operating model from scratch every time the frontier changes. A workflow that feels barely possible today may become ordinary in a few months. A task that needs heavy human supervision now may become safe to delegate later if the context, permissions, evals, and review paths are already in place.

That matters internally and externally.

Internally, it means the CTO has to keep pushing the boundary of what the company asks AI to do. Some workflows may be too expensive, too brittle, or not quite possible today. But in this market, today’s bad fit can become tomorrow’s normal operating path. The company needs someone continuously testing that boundary, so it is ready to absorb the next jump in model capability.

Externally, it means building products and services with the same awareness. If you are building an AI-native product, the roadmap cannot only reflect what models can do this week. It has to account for what they are likely to make possible next, and where your product should be ready to absorb that capability.

This is not about guessing the future perfectly. It is about building a company that can adapt when the future arrives faster than expected.

My current view: the CTO’s job is not to be the smartest AI user in the company. It is to make the work itself smarter: sharper questions, clearer communication, stronger reviews, sounder decisions, and more consistent follow-through across the team. That is where productivity compounds.

Originally published on LinkedIn.

This guide is outcome of that migration. React front-end was quite simple and do not have much moving parts so it’s quite simple to do this. If you are looking to deploy monolith application then you can read the following posts I have written:

• Deploy Rails app and Postgres with Kamal(previously MRSK) on single DigitalOcean server
• How to deploy multi-environment(staging, production) application using Kamal(previously MRSK)
• How to deploy a NodeJS application using Kamal (previously MRSK)

Kamal Config

First, let’s create a simple deploy.yml for Kamal. The configuration is mostly boilerplate. I am using GHA cache to speed up the deploys.

# config/deploy.yml

service: react-app
image: username/image-name

servers:
  web:
    - 123.456.789.012

registry:
  username: docker_username
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    FOO: BAR
    NPM_CONFIG_PRODUCTION: false

builder:
  multiarch: false
  cache:
    type: gha
    options: mode=max

If you wish to, you can use “registry” cache as well. I have noticed that it’s faster by 5-10 seconds but if you are using free Docker Hub account then you only get one free image. That’s why I prefer to use “gha” as cache back-end. Kamal only supports “gha” and “registry”. If you would like to use the “registry” cache, use the following builder config:

builder:
  multiarch: false
  cache:
    type: registry
    options: mode=max,image-manifest=true,oci-mediatypes=true

Dockerfile

Our app is using quite old version of NodeJS. This is why I like Kamal more than anything. No platform dependency or requirement to fulfil. You create your own environment and Kamal provides thin wrapper around the Docker commands. As simple as it could be.

Here, we first create a build step. Once the build is ready, no need to include all those node_modules in our final application. We keep it as small as possible so that it’s easier to perform operations.

In our app, we have a small Express app that serves the static file. That’s why I have created a separate package.runtime.json file. You can modify that part as per your own need. The next session will explain how the express server works.

# Dockerfile

FROM node:11.10.1 as build

WORKDIR /app

ENV NODE_ENV="production" \
    NPM_CONFIG_PRODUCTION="false"

COPY . .

RUN npm ci

RUN npm run build

FROM node:11.10.1

WORKDIR /app

ENV NODE_ENV="production" \
    NPM_CONFIG_PRODUCTION="false"

COPY --from=build /app/build ./build
COPY --from=build /app/server.js ./server.js
COPY --from=build /app/package.runtime.json ./package.json

RUN npm install --production

EXPOSE 3000

CMD ["node", "server.js"]

Express Server

We have a tiny express server that serves the static files with appropriate headers. We have configured Cloudflare to provide SSL support and cache the static assets. Only index.html file is not cached, and bundled assets will be served by this Express server only once. Then Cloudflare will take the load. All of this is deployed on our $4 DigitalOcean droplet and it is handling our moderately busy app very well.

I have configured the Webpack to generate bulid with hash, so with every new build, the new hash will rename the file - same as Propshaft.

I have added a route for Kamal healthcheck as well even though the health-check step has been removed in Kamal 1.6.0.

// server.js

const compression = require('compression');
const express = require('express');
const path = require('path');
const port = process.env.PORT || 3000;
const app = express();

// Use compression middleware
app.use(compression());

// Serve static files with cache control headers
app.use(express.static(path.join(__dirname, 'build'), {
  maxAge: '1d', // Cache for 1 day
  setHeaders: (res, path) => {
    // Set cache-control headers based on file types
    if (path.endsWith('.js')) {
      res.setHeader('Cache-Control', 'public, max-age=31536000'); // 1 year
    } else if (path.endsWith('.css')) {
      res.setHeader('Cache-Control', 'public, max-age=31536000'); // 1 year
    } else if (path.endsWith('.html')) {
      res.setHeader('Cache-Control', 'public, max-age=0'); // No cache
    } else {
      res.setHeader('Cache-Control', 'public, max-age=86400'); // 1 day
    }
  }
}));

// Kamal health check route
app.get('/up', (req, res) => {
  res.send(`<!DOCTYPE html><html><body style="background-color: green"></body></html>`);
});

// Send all requests to index.html to handle routing in React Router
app.get('*', (req, res) => {
  res.sendFile(path.resolve(__dirname, 'build', 'index.html'));
});

// Start the server
app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

We only need ExpressJS and the compression packages to run this small server. That’s why we are including only these two packages in the runtime package file.

package.runtime.json

{
  "name": "your-app",
  "version": "1.0.0",
  "main": "server.js",
  "dependencies": {
    "compression": "^1.7.4",
    "express": "^4.16.4"
  },
  "scripts": {
    "start": "node server.js"
  }
}

GitHub Action

It’s a headache to run the deployment from your own machine. The following GitHub Actions config will solve it for you. I have added a concurrency config too that will let only one deploy run at a time. It’s useful when you make multiple commits in a short time.

# .github/workflows/deploy.yml

name: Deploy to production

# To make sure that only one deploy runs at a time. Deploy lock will not let simultaneous deployments.
concurrency:
  group: ${{ github.workflow }}

on:
  push:
    branches: [master]

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      KAMAL_REGISTRY_PASSWORD: $

    steps:
      - name: Set up Docker Buildx for cache
        uses: docker/setup-buildx-action@v3

      # Since we are using GHA cache, we need to expose the cache to the runtime
      - name: Expose GitHub Runtime for cache
        uses: crazy-max/ghaction-github-runtime@v3

      - name: Checkout code
        uses: actions/checkout@v3

      # Ruby is only needed to install Kamal
      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          ruby-version: '3.3'

      - name: Install dependencies
        run: gem install kamal

      # This is to facilitate Kamal with the private key to access server(s)
      - uses: webfactory/ssh-agent@v0.7.0
        with:
          ssh-private-key: $

      - name: Run deploy command
        run: kamal deploy

Understanding the Purpose of Rails’ schema.rb File

The schema.rb file is essentially a blueprint of your Rails application’s database. Imagine your app as a house - the schema.rb is the floor plan you’d show your architect. It outlines your database’s structure, detailing tables, columns, their types, primary keys, and even relationships between them. Migrations update and generate this file automatically.

Why is it important? Think of it this way: a new team member joins and needs to understand your database structure. Rather than making them trawl through countless migrations (a process as thrilling as watching paint dry), they can reference the schema.rb file. This file provides a concise summary of your database structure, the Cliff Notes version, if you will.

The schema.rb file in a new Rails application contains the following comment:

# This file is auto-generated from the current state of the database. Instead  
# of editing this file, please use the migrations feature of Active Record to  
# incrementally modify your database, and then regenerate this schema definition.  
#  
# This file is the source Rails uses to define your schema when running \`bin/rails  
# db:schema:load\`. When creating a new database, \`bin/rails db:schema:load\` tends to  
# be faster and is potentially less error prone than running all of your  
# migrations from scratch. Old migrations may fail to apply correctly if those  
# migrations use external dependencies or application code.  
#  
# It's strongly recommended that you check this file into your version control system.

Decoding the Authority Over the Database in Rails

The Rails docs’ interpretation of the authority over the database schema has seen notable changes over the years. To illustrate this, let’s look at two significant commits on the Rails GitHub.

The first version of the docs from 2012 claimed that the db/schema.rb or an SQL file generated by Active Record was the authoritative source for your database schema.

What are Schema Files for?

Migrations, mighty as they may be, are not the authoritative source for your database schema. That role falls to either db/schema.rb or an SQL file which Active Record generates by examining the database. They are not designed to be edited, they just represent the current state of the database.

There is no need (and it is error prone) to deploy a new instance of an app by replaying the entire migration history. It is much simpler and faster to just load into the database a description of the current schema.

However, the follow-up version from 2018 which we read today clearly states that your actual database remains the authoritative source.

What are Schema Files for?

Migrations, mighty as they may be, are not the authoritative source for your database schema. Your database remains the authoritative source. By default, Rails generates db/schema.rb which attempts to capture the current state of your database schema.

It tends to be faster and less error prone to create a new instance of your application’s database by loading the schema file via rails db:schema:load than it is to replay the entire migration history. Old migrations may fail to apply correctly if those migrations use changing external dependencies or rely on application code which evolves separately from your migrations.

The commit was made with the following comment from the contributors:

    Update schema.rb documentation [CI SKIP]

    The documentation previously claimed that `db/schema.rb` was “the
    authoritative source for your database schema” while simultaneously
    also acknowledging that the file is generated. These two statements are
    incongruous and the guides accurately call out that many database
    constructs are unsupported by `schema.rb`. This change updates the
    comment at the top of `schema.rb` to remove the assertion that the file
    is authoritative.

    The documentation also previously referred vaguely to “issues” when
    re-running old migrations. This has been updated slightly to hint at the
    types of problems that one can encounter with old migrations.

    In sum, this change attempts to more accurately capture the pros, cons,
    and shortcomings of the two schema formats in the guides and in the
    comment at the top of `schema.rb`.

    [Derek Prior & Sean Griffin]

While db/schema.rb and migrations play vital roles in managing your database structure, neither are the definitive descriptors of your schema. The db/schema.rb file is a Rails-generated snapshot of your database structure, and is useful for setting up new instances of your database quickly. Migrations, however, are used to implement incremental changes to your database over time.

As developers, ensuring your migrations are up-to-date and error-free is paramount. These migrations should be able to recreate db/schema.rb accurately. In the event of any breaking changes, such as class name alterations, best practices must be followed to ensure your migrations remain reversible and unaffected by such changes. Understanding these principles will help manage your database schema effectively in Rails.

Working with schema.rb 

While it might be tempting to manually modify schema.rb, resist the urge! Any changes should be made through migrations. This ensures that the schema.rb file can be regenerated correctly.

When working on a new or unreleased project, it might make sense to adjust existing migrations rather than creating new ones. This way, you keep the migration history lean and focused. However, if a project is already in production, creating new migrations for each database change is the standard practice. This ensures that your database changes are properly tracked, and each team member can understand when and why the database structure was changed.

In new and unreleased projects, I often opt to modify existing migrations rather than creating new ones. In older projects where migrations are being rectified, I use a custom rake task to streamline the process. Here’s a snippet of the task for resetting the database by dropping the schema:

# lib/tasks/complete_reset.rake

namespace :db do
  desc 'reset the database by dropping the schema'
  task complete_reset: :environment do
   raise unless Rails.env.local?
    
    FileUtils.rm_f('db/schema.rb')
    Rake::Task['db:drop'].invoke
    Rake::Task['db:create'].invoke
    Rake::Task['db:migrate'].invoke
    Rake::Task['db:seed'].invoke
    Rake::Task['dev:prime'].invoke
  end
end

This task handles the removal of db/schema.rb, drops the database, creates it anew, migrates and seeds it, and primes it for development. Be extremely cautious when choosing the environment this task is applied to; it should not be used on deployed servers once your app has hit staging or production stages.

This task becomes particularly useful in the early stages of a project when specifications are frequently changing. When switching between branches, conflicts in the schema.rb file can often arise due to these rapid changes. This task helps you avoid creating excessive migrations and ensures a smooth, efficient development process by resolving such conflicts.

It’s important to note that this task also invokes another task, dev:prime. This primes the development database with test data, providing a consistent starting state after a reset. So, firing complete_reset on the development environment not only clears any schema conflicts, but also populates your database with test data for a fresh start.

FAQs

Should I modify the schema.rb file manually?

As a rule of thumb, you should not manually modify the schema.rb file. Instead, you should use ActiveRecord Migrations to alter your database schema. This ensures that changes are properly tracked and the schema.rb is updated correctly. In rare, exceptional cases with legacy code or failed migrations, direct modification could be considered but it’s a risky approach and should only be done with extreme caution.

What are some best practices for managing the schema.rb file in a team environment?

1. Version Control: Always keep the schema.rb file in your version control system to keep track of its changes over time.
2. Don’t Modify Manually: Avoid making manual modifications to the schema.rb file. Use ActiveRecord migrations instead.
3. Review Changes: Before committing changes to the schema.rb file, review them to make sure they align with the changes made in your migrations.
4. Sync with Database: Always make sure your schema.rb file is synchronized with the current state of your database schema.

What happens if the schema.rb file is deleted or modified?

If the schema.rb file is deleted or modified manually, it may lead to inconsistencies between the actual state of your database and the Rails application’s understanding of the database schema. This can lead to unexpected errors or bugs in your application. Therefore, it is advised not to delete or manually modify the schema.rb file.

Traefik dashboard can run in two modes - secure and insecure. Traefik recommends secure mode but this post wil cover how to enable both the modes.

First, let’s see the unsecure mode. Unsecure mode is very easy to configure. It is unsecure because it exposes Traefix API on the entrypoint . It means that after configuring the dashboard in unsecure mode, the dashboard will be available on the port 8080 of the host.

Adding the following snippet in the config/deploy.yml file will enable the unsecure dashboard.

# config/deploy.yml
traefik:
  options:
    publish:
      - 8080:8080
  args:
    api.dashboard: true
    api.insecure: true

After adding the configuration, run the kamal traefik reboot command to apply the configuration. This command will stop, remove and start new container again with the latest configuration. The dashboard should be visible on the port 8080 of the host server now e.g. http://99.99.99.99:8080

Now, let’s see about the secure mode. It’s called secure mode because the API is not expose on the entrypoint. We need to create a router rule that uses api@internal service. Let’s look at the configuration for the secure model.

traefik:
  options:
    publish:
      - 8080:8080
  args:
    api.dashboard: true
  labels:
    traefik.enable: "true"
    traefik.http.routers.dashboard.rule: Host(`traefik.example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
    traefik.http.routers.dashboard.service: "api@internal"
    traefik.http.routers.dashboard.middlewares: "auth"
    traefik.http.middlewares.auth.basicauth.users: test:$2y$05$H2o72tMaO.TwY1wNQUV1K.fhjRgLHRDWohFvUZOJHBEtUXNKrqUKi

Here, we have configured Traefik dynamically with help of Docker labels. First of we have created a router. Then we attached the router with the api@internal service because in the secure mode we have to do this manually. After that, we added the auth middleware to Trafeik. In the last, we conifgured this auth middleware to use HTTP Basic Authentication and provided it with the credentials. You can read more about the rules in the details on the Traefik docs .

The credentials are in the “username:hashed_password” format. The credentials are generated with the htpasswd command. Let’s say you want to create a user with the username “admin” and the password “super_strong_password” then you can use the following command:

htpasswd -nb admin super_strong_password
# output: admin:$apr1$2FGO09Gu$PSZdmmJqyrXWYvidWAm6p0

You will get the password hash in the output. Just copy paste the output with the username:password in the labels. The official Traefik docs mention that you need to escape the $ character but you don’t need if you are using Kamal but Kamal escapes the $ sign for these labels.

That’s it! Don’t forget to reboot the Trafeik container with the kamal traefik reboot comamnd. After that, the dashboard should be accessible on the http://traefik.example.com/dashboard endpoint.

P.S. I would not recommend running the dashboard permanently if you are using a small single server with multple apps. Because that would eat up the valuable server resources.

First of all, let’s generate an express app. I installed express-generator and generated a new app using the express express-app command.

Then let’s create a Dockerfile:

FROM node:16

# Create app directory
WORKDIR /usr/src/app

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

RUN npm install
# If you are building your code for production
# RUN npm ci --omit=dev

# Bundle app source
COPY . .

EXPOSE 3000
CMD [ "node", "bin/www" ]

Now, let’s use Kamal to generate the config files. Make sure you have Kamal installed on your machine. If not, you can follow these instructions on README.

Let’s init the configuration with kamal init. It will create config/deploy.yml and .env files in your root directory. My config file looks like this:

service: express-app

image: user/express-app
servers:
  - 146.190.86.77

registry:
  username: user
  password:
    - KAMAL_REGISTRY_PASSWORD

healthcheck:
  path: /
  port: 3000

The config file I have created here is minimal. It defines a service, provides the server IP, Docker Hub registry info and healthcheck information. After successful deploy, Kamal will ping the “/” path on port “3000” and will expect the “200 OK” response from the server.

Now, let’s deploy the app with the kamal deploy command. Kamal will deploy your app and you can check if it is live or not by entering the server IP address in the browser. If you want to connect your app with the database, or host multiple environment of the application like staging, production then you can check out my previous posts . They are platform agnostic and applicable on NodeJS apps as well.

Kamal uses Traefik as a reverse proxy. It means that any incoming request will be handled by Traefik. Traefik will handover the request to appropriate server and Docker container based on the configuration. For example, let’s assume that we have deployed the staging and the production version of our apps on the server. Then we have to configure Traefik in such a way that it points “staging.myapp.com” and “production.myapp.com” to the staging and the production Docker containers respectively.

Kamal uses Traefik with “docker” as a provider. Traefik rules are configured by providing certain lables to Docker containers. You can read more about it in their docs.

Let’s start configuring Kamal. Kamal has a feature called “destination”, we will use that to create two separate destinations - “staging” and “production”. First, we will create the config/deploy.yml file with common options like service name, image name, registry, common environment variables, and Postgresql database as acessory.

# config/deploy.yml
service: myapp
image: user/myapp

registry:
  username: user
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    DB_HOST: 99.99.99.99
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:15
    host: 99.99.99.99
    port: 5432
    env:
      clear:
        POSTGRES_USER: 'myapp_db_user'
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

Staging config

Now, let’s create a destination named “staging” with the config/deploy.staging.yml file.

# config/deploy.staging.yml
servers:
  - 99.99.99.99

labels:
  traefik.http.routers.myapp-web-staging.rule: Host(`staging.myapp.com`)

env:
  clear:
    POSTGRES_DB: myapp_staging
    IS_STAGING: true

In this file we have declared the server for the staging environment, environment variable spcific to the environment and we have used the labels key that will attach the labels to the staging Docker container.

Let’s understand these rules with little more details. Kamal applies some default Traefik labels to each container in the “service-role-destination” format. With the help of these labels Traefik defines a service (not to be confused with Kamal service). With the rule mentioned in the deploy.staging.yml file above, we are overriding the default labels. Since we have not specified any role, Kamal will assign the web role to the service. Anyways, there has to have one “web” role if we are specifying roles.

Don’t forget to replace “staging.myapp.com” with your domain. The domain should be configured to point to the IP of the server.

Production config

# config/deploy.production.yml
servers:
  - 99.99.99.99

labels:
  traefik.http.routers.myapp-web-production.rule: Host(`rails.myapp.com`)

env:
  clear:
    POSTGRES_DB: myapp_production

The deploy.production.yml is similar to the staging configuration. Since we are going to deploy both the staging and the production as well as Postgresql(the database) on the same server, the same IP is configured under the hosts key. Again, don’t forget to configure your domain.

That’s it! You should have both the staging and the production as well as the database deployed on the same or different server(s) and accessible from the domains you have set for each environment!

In this post, I am not getting into how Kamal works and the benefits of using it. DHH has created this fantastic introduction video for that.

Kamal is capable to deploy applications on multiple servers as well as a single server. In this post, I will describe how we can deploy a Rails application and Postgresql on a single VPS.

First of all, install and init Kamal on your local machine with instructions on the README.

The key to the single server setup is to use the same hosts for the web as well as accessories. Here is what the deploy.yml file looks like for that setup:

service: my-app
image: user/my-app

servers:
  web:
    - 134.209.111.91

registry:
  username: tannakartikey
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    DB_HOST: 134.209.111.91
  secret:
    - RAILS_MASTER_KEY
    - POSTGRES_PASSWORD

accessories:
  db:
    image: postgres:15
    host: 134.209.111.91
    port: 5432
    env:
      clear:
        POSTGRES_USER: 'my_app'
        POSTGRES_DB: 'my_app_production'
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

I am using the default Dockerfile that is generated with Rails 7.1. Create and place the Dockerfile in your root directory if it does not exist already. You also need to create a .env file in the root of the repository and also need to make minor changes in your config/database.yml file.

I have created this sample repository that has all the code described in this post. This commit holds all the Kamal setup-related changes.

Be sure NOT to include your .env file in Git or your Dockerfile. If you are using Docker Hub or any other registry, make sure to make the image private because it holds a copy of the application code.

Next, we need to set up the server since we are deploying to the server for the first time. We can do it with:

bin/kamal setup

The setup command will set up all the accessories and deploy the app on the server.

That’s it! You should have a running Rails app on the server which is connected to Postgresql on the same server. For subsequent deploy you can use kamal deploy or kamal redeploy based on your needs.

I have been using the USD4 droplet on DigitalOcean. If you are using a similar configuration I would recommend creating a swap partition on the server. I am assuming that if you are doing this, you are not expecting loads of traffic on day one. But without a swap partition, some simple operations like migrating the database may also fail.

It can also be possible to host the same code multiple times for different environments i.e. staging, production etc. For that, the “role” and “destination” functionality of Kamal can be used. I might cover that in a separate post.

File.foreach method reads the file line by line; that is why it is safe to use for large files.
It can accept the block to execute each line of a file.

Example:

File.foreach('example.jsonl') { |line| JSON.parse(line) }

However, in the Ruby documentation, you will not find this method defined on the File class. It is defined on IO, which is a superclass of File.

To download the large files from the Internet

We can use IO.copy_stream to download the large files from the Internet. It will create a stream instead of loading the whole file into memory before writing.

Example:

IO.copy_stream(open('http://example.com/file.jsonl'), 'file.jsonl')

To sum it up…

Memory friendly methods:
File.foreach, IO.copy_stream

Other methods to read a file:
File.read, File.readlines