블로그

OpenAI's Sol vs Codegiant: Scored on COBOL Migration

OpenAI's Sol scored 47.4% across 43 COBOL migration and platform criteria. We ran it and Codegiant through the same modernization to show you why.

게시일

2026년 11월 12일

읽는 시간

21분

The Codegiant team gave OpenAI's Sol the complete CardDemo batch estate and asked it to modernize the system, from analysis through proof. The run used GPT-6.1 Sol, with 78 files of COBOL, copybooks, JCL, a PROC, a control card and sample data. We checked what it delivered against the original source and Codegiant's deterministic facts.

OpenAI reports 97.9% accuracy at higher reasoning effort on its own COBOL migration evaluation, where a migration succeeds only if every hidden test passes. That is OpenAI's result on its evaluation, not the result of this CardDemo run. OpenAI also says production use still requires expert review and testing against business rules and dependencies.

Every point below comes back to one thing: passing the cases a model ran is not the same as delivering a complete modernization. Sol kept the posting logic and quirks on the cases it tested, but invalid digits could post as zero, the status-decode branch was never converted, and no I/O error path was triggered. Codegiant's comparison covers parsed dependencies, typed data, compiled jobs and independent proof.

Across the 43 scored criteria, Sol earned 204 of 430 points, or 47.4%, and Codegiant earned 430 of 430. That combined score includes 35 criteria for the work in the prompt and eight additional platform criteria. The asked-items subtotal and the platform subtotal are shown separately, so capabilities outside the prompt are not confused with failures to follow it.

What follows is a summary, what we tested, the headline scores and a comparison at a glance, then the full prompt and detailed findings. The complete 43-row scorecard remains below the findings, followed by the nine areas where modernization breaks beyond the prompt and the bottom line.

Summary

Analysis and dependencies. OpenAI's Sol counted the inventory correctly, but its analysis captured 21% of DD-to-dataset bindings, 14% of program-to-file links and no copybook-to-program links. Codegiant derives those relationships from parsed source, with a line behind every link.
The map. Sol's separately delivered map got the basic links and cited source lines right, but included 148 uncertain edges and left most job hand-offs uncertain. Codegiant builds the map deterministically and includes utility producers and gated conditions.
Converted code and data. Sol preserved posting logic and quirks on the cases it ran, but invalid digits could post as zero and PostgreSQL held each record in one bytea column. Codegiant keeps mainframe number and error behaviour, with named Java fields and typed database columns.
Jobs and proof. Sol delivered the POSTTRAN step, DDs and return codes by hand, then compared outputs with a local COBOL compiler. Its proof covered 65% of decisions and triggered no I/O error paths. Codegiant generates jobs from JCL and adds decision, fault and production-replay checks.
Beyond the prompt. Sol did not deliver an integrated deployment and operations platform in this run. Codegiant includes one-click deployment, managed database deployment, DevSecOps, AI SRE, integrations, builders, governance and self-hostable Kronos models. These eight platform criteria are scored separately from the work requested.
Scores. Across 43 criteria worth 430 points, Sol scored 204/430 (47.4%) and Codegiant 430/430 (100%). For the 35 asked-item criteria alone, the scores are 204/350 and 350/350. For the eight additional platform criteria, they are 0/80 and 80/80.

What we tested

We supplied GPT-6.1 Sol with the complete CardDemo batch archive: 78 files, including COBOL programs, copybooks, JCL jobs, a PROC, a control card and sample data. The prompt asked for the whole system to be modernized to Java 21 and PostgreSQL, with POSTTRAN completed end to end first if the model could not finish everything.

The analysis was checked against Codegiant's deterministic facts for those files, and Sol's dependency map was checked edge by edge against Codegiant's graph. The detailed findings below cover the CBTRN02C posting slice, its datasets, POSTTRAN and the proof Sol supplied. The full prompt is reproduced below the overview.

OpenAI's published evaluation is a separate benchmark with hidden tests requiring byte-for-byte output parity. Its reported 97.9% is not a score on CardDemo or on this article's criteria. This article's scores are Codegiant's assessment of the supplied run and the compared platform capabilities, not an independent replication of OpenAI's benchmark.

How Codegiant scores against OpenAI's Sol

Each of the 43 criteria carries up to 10 points, with partial scores shown in the complete scorecard. The combined score includes eight platform criteria beyond the prompt. Sol's zero on those criteria means the capabilities were not delivered in this run, not that the model cannot generate related code. Codegiant's platform scores reflect the capabilities documented in the published Anthropic and AWS comparisons.

OpenAI's Sol	Codegiant
47.4% (204/430 points)	100% (430/430 points) ✅

Asked items: Sol 204/350, Codegiant 350/350. Additional platform criteria: Sol 0/80, Codegiant 80/80. These scores use this article's criteria, not OpenAI's hidden-test evaluation.

OpenAI's Sol vs Codegiant: the comparison at a glance

This overview summarizes the findings without replacing them. Every original analysis table, all 43 scored rows, both subtotals and the nine beyond-the-prompt sections remain in full below.

Area	OpenAI's Sol	Codegiant
Analysis	Counts and basic facts pass; DD bindings reach 21%, program-to-file links 14% and copybook use 0% in the analysis.	The parser resolves inventory, layouts, DD bindings, copybook use and calls to source lines.
Dependency map	Basic links and cited lines pass; 148 edges are uncertain, utility suppliers are missed and only 13% of job hand-offs are certain.	The deterministic graph includes producers, consumers and conditions, with source lines behind the relationships.
Java conversion	Posting and quirks pass on tested cases, but invalid digits post as zero, status decoding is absent and Java uses raw byte offsets.	Named types and fields preserve number, status and error behaviour, with blocks traced to COBOL.
PostgreSQL data	Keys, order and alternate index pass; each record is stored in one bytea column and loaded unchecked.	Copybook fields become typed columns, mainframe order is kept and records are checked before cutover.
POSTTRAN	Step, DDs, reject generation and return codes pass in hand-written job handling.	The job is generated from JCL, including catalogue behaviour and return codes.
Proof	Outputs are compared with a local compiler; decision coverage is 65%, with no I/O faults or bad-data cases triggered.	Decision outcomes, error paths and field boundaries are checked, alongside captured production replay.
Platform beyond the prompt	The eight integrated platform capabilities were not delivered in this run.	Deployment, managed database deployment, DevSecOps, AI SRE, integrations, builders, governance and private hosting are included.
Beyond the demo	The brief identifies further issues around context, uniformity, site settings, production data, batch windows, operations, failures, proof and scale.	The nine detailed sections compare those risks with estate-wide definitions, mainframe behaviour, restart, independent proof and production operations.

The prompt given to GPT-6.1 Sol

I am attaching carddemo-batch.zip. It is the complete batch system of a small credit card company running on an IBM mainframe: COBOL programs, copybooks, JCL jobs, a PROC, a control card and sample data files exported from VSAM and sequential datasets.

We are moving this system off the mainframe. Please do the full modernization:

Analyse the whole system first. Tell us what is in it, how the programs, jobs and datasets fit together, and what the business logic is.

Build a complete dependency map of this CardDemo batch system from the files you were given. Include every element you can find (jobs, steps, programs, copybooks, datasets, utilities, control cards, procedures, called routines and anything else) and every relationship between them. Deliver two files:

dependency-map.json: a list of nodes (id, type, name) and a list of edges (from, to, type, direction where it applies). For every node and edge, give the source file and line it comes from.

dependency-map.html: one self-contained page that draws the same graph, where clicking a node shows its edges and source lines.

Do not leave anything out to keep the map small. If a relationship is uncertain, include it and mark it uncertain.

Convert the COBOL programs to Java 21.

Move the VSAM datasets to PostgreSQL. Design the schema and load the sample data into it.

Convert the JCL jobs to Java batch jobs, so that operations can still run the same jobs, in the same order, with the same results.

The new system must behave exactly like the old one. Keep every existing behaviour, including odd or buggy-looking behaviour, byte for byte. Do not fix bugs and do not add new features. If you find something that looks like a defect, keep it and list it in your notes.

Prove it. Compile and run the original COBOL with GnuCOBOL, build and run your Java, feed both the same inputs, and compare every output: the output files, the database or dataset contents after the run, the return codes and the console output. Do this for every program and job you converted. Report the result per program and per job: identical, different (show the differences), or not compared (say why). If any COBOL program does not compile or run under GnuCOBOL, say so and explain what you did about it. Do not hide or round away differences.

Convert every program and every job in the archive. If you cannot finish everything, finish the nightly transaction posting job (POSTTRAN) end to end first, with every dataset it reads and writes, including its equivalence check, and then continue with the rest.

Deliver one zip file with this layout:
analysis/       your analysis of the system and its business rules
src/            the Java project (Maven), building with one command
db/             the PostgreSQL schema and the data load
jobs/           the batch jobs and how to run them
tests/          your tests
proof/          the GnuCOBOL setup, the inputs you used, the scripts that run both sides and compare them, the raw outputs of both sides, and an equivalence report with one line per program and per job
README.md       how to build and run everything from a clean machine, including how to rerun the equivalence check
NOTES.md        everything you could not do, every assumption you made, every defect you kept, and every question you would ask us
Ask us if anything about the old system is unclear before you assume.

1. Analysis: Sol against our deterministic facts

Fact family	GPT-6.1 Sol	Codegiant
Inventory	Pass. Counts right, but they come from reading the folder, not from a parse	Every program, copybook, job, PROC and data file, counted from the parsed source
Record lengths and keys	Pass. Typed by hand from the copybooks into a table	Every record and key taken from the copybook and the DEFINE, deterministic
Steps and what each runs	Pass. The EXEC lines copied out of the JCL	Every step resolved to its program or utility, with PROC steps expanded
Datasets	Pass. Names copied as text	Every dataset as a node in the graph, with its data and index parts
Which DD binds which dataset	21%. Continuation lines pasted as bare `DSN=` text with the DD name lost	Every reference bound to its DD, step and line
Disposition per DD	39%. The rest left as "see the original JCL"	Every disposition resolved per DD, defaults included
Program to file links	14%. Only the six POSTTRAN files; the rest in prose	Every SELECT tied through its DD to the dataset, with the open mode
Copybook use	0%. No copybook-to-program link at all	Every COPY link, so a layout change shows every program it touches
Calls	80%. The abend routine used by 11 programs is not named	Every call site, internal and system
Job order through data	Partial. Prose chains, and one statement contradicts its own table	Every hand-off between jobs, with the producing and consuming lines

2. The dependency map

Sol's map was checked edge by edge against our graph for the same files.

Check	GPT-6.1 Sol	Codegiant
Source lines behind each link	Pass. Every cited line checks out	Every node and edge, with its line
COPY, CALL, step to program, program to dataset	Pass. The basic links are complete, with generations kept	Every link built deterministically, and rebuilt the same way on every change
Job order through data	13% certain. Most hand-offs flagged "uncertain"	Every hand-off stated, with both lines
Datasets created or loaded by utilities	Missed as suppliers. The GDG definitions, backup and reload jobs ahead of POSTTRAN are not linked	Every producer, utility or program
IDCAMS conditions	Listed only. No link from a condition to the command or return code it gates	Every condition tied to what it gates
Guessed links	148 uncertain edges. A program with no job gets datasets matched by DD name from other jobs	Proven links only
Usable view	Hairball. Fields, paragraphs and spool lines in one graph; the drawn overview covers one job	Every job, step and dataset navigable, with a description on click
Repeatable	One-off. A model's file; the next run can differ	Same input, same map, every time

3. CBTRN02C to Java (asked)

Construct	GPT-6.1 Sol	Codegiant
Validation and posting	Pass. Identical on the cases it ran	Identical, proven on every path
Quirks	Pass. All kept	All kept, because every statement is converted as written
Signed amounts	Pass on clean data. Sign read from a hand-written nibble test	Every sign rule of the compiler reproduced exactly
Invalid digits	Fail. Spaces read as 0, and the bad data posts	A data exception, as on the mainframe
Arithmetic	Pass. Truncation written by hand at each statement	Exact decimal arithmetic from the target picture, one rule everywhere
Timestamp REDEFINES	Pass. A Java formatter that copies the layout	The view over the same bytes, exactly as defined
FILE STATUS and the status decode	Fail. Four fixed codes; the decode branch was never converted	The real status for every outcome, decode included
I/O failures	Partial. Four codes; one path stops where the COBOL continues	The same behaviour as the COBOL on every failure
Java form	Fail as modernization. Raw byte offsets, no field names	Named record types and fields, every block traced to its COBOL line

4. The slice's datasets in PostgreSQL (asked)

Check	GPT-6.1 Sol	Codegiant
Fields as columns	0%. Each record dumped into one `bytea`	Every copybook field as a typed column
Keys and order	Pass. EBCDIC byte order on a generated key	Typed keys, mainframe order kept
Alternate index	Pass. An expression index on the bytes	Same lookups and the same order as the VSAM path
Load	Pass. Bytes copied in unchecked	Every record verified before cutover

5. POSTTRAN as a job (asked)

Check	GPT-6.1 Sol	Codegiant
Step, DDs, program	Pass. Written by hand in a shell switch	Generated from the JCL
DALYREJS(+1), NEW,CATLG,DELETE	Pass	Catalogue behaviour as on z/OS
Return codes	Pass	As on z/OS

6. Proof of the slice (asked)

Check	GPT-6.1 Sol	Codegiant
Method	Pass. Full output compared against a local compiler	Full output compared, plus replay of captured production runs
Decision coverage	65%. Happy paths and the four reject reasons only	Every decision outcome
Error paths	0%. No open, read, write or close failure was ever triggered	Every error path triggered and compared
Bad data	0%. No invalid digits, odd signs or LOW-VALUES	Field boundaries, a census of the real data, missing and locked datasets

7. The complete scorecard: asked items and platform

Rows 1–35 retain the original asked-items benchmark and scores. Rows 36–43 add the platform comparison beyond the prompt: Sol receives 0/10 for capabilities not delivered in this run, not a claim that it cannot generate related code; Codegiant receives 10/10 based on the platform capabilities described in our published Anthropic and AWS comparisons. The combined total includes both scopes.

#	Item	GPT-6.1 Sol	Codegiant	Sol score	Codegiant score
1	Inventory, records, keys	Pass, typed by hand	Parsed, deterministic	9/10	10/10
2	Steps, programs, datasets	Pass, copied as text	Every node in the graph	9/10	10/10
3	DD to dataset bindings	21%, DD names lost	Every binding with its line	4/10	10/10
4	Dispositions	39%	Every one resolved	4/10	10/10
5	Program to file links	14%	Every one, with the open mode	4/10	10/10
6	Copybook use	0%	Every link, with its impact	1/10	10/10
7	Calls	80%	Every call site	7/10	10/10
8	Job order through data	Partial, contradicts itself	Every hand-off	5/10	10/10
9	POSTTRAN business logic	94%	Every rule checked against its statements	9/10	10/10
10	Business logic, other programs	93%, CBTRN03C wrong	Every program as written	8/10	10/10
11	Map: source lines behind each link	Pass, every line checks out	Every node and edge	9/10	10/10
12	Map: copybook, call, step and dataset links	Pass, basic links complete	Every link, built deterministically	9/10	10/10
13	Map: job order through data	13% certain	Every hand-off stated	2/10	10/10
14	Map: datasets built by utilities	Missed as suppliers	Every producer linked	4/10	10/10
15	Map: IDCAMS conditions	Listed only	Tied to what they gate	2/10	10/10
16	Map: guessed links	148 uncertain	Proven links only	5/10	10/10
17	Map: usable view	Hairball	Navigable, described on click	5/10	10/10
18	Map: repeatable	One-off file	Same input, same map	3/10	10/10
19	Real defects found	Pass, three found	Every defect surfaced, with its line	9/10	10/10
20	Posting logic and quirks	Pass on the cases run	Proven on every path	9/10	10/10
21	Invalid digits	Fail, garbage posts as 0	A data exception, as on the mainframe	3/10	10/10
22	Arithmetic	Pass, hand-written per statement	One exact rule everywhere	9/10	10/10
23	Timestamp layout	Pass, copied by a formatter	The view as defined	9/10	10/10
24	FILE STATUS and decode	Fail, four fixed codes	The real status for every outcome	3/10	10/10
25	I/O failure behaviour	Partial	As the COBOL	4/10	10/10
26	Java form	Fail, raw byte offsets	Named types and fields, traced to the COBOL	2/10	10/10
27	Fields as columns	0%, one `bytea` per row	Every field typed	0/10	10/10
28	Keys, order, alternate index	Pass	Typed keys, mainframe order	8/10	10/10
29	Load	Pass, unchecked	Every field checked	7/10	10/10
30	POSTTRAN step, DDs, return codes	Pass, hand-written	Generated from the JCL	9/10	10/10
31	Reject generation	Pass	Pass	9/10	10/10
32	Proof method	Pass, local compiler only	Plus production replay	8/10	10/10
33	Decision coverage	65%	Every outcome	5/10	10/10
34	Bad data and faults	0%	Every boundary and fault	2/10	10/10
35	README, NOTES, rerunnable proof	Pass	Complete, with a certificate per program and job	9/10	10/10
36	One-click deployment	Not delivered in this run	One-click deployment to AWS, Google Cloud, Azure, Oracle Cloud, DigitalOcean, Cloudflare or Fly.io, on-premises, or Kubernetes	0/10	10/10
37	Managed database deployment	Schema and load supplied; managed database deployment not delivered	Tables go to the client's own managed database, deployed with the app	0/10	10/10
38	Included DevSecOps	Integrated toolchain not delivered	Git, CI/CD, Terraform, code review, docs, observability, error and performance monitoring, uptime checks and status pages included	0/10	10/10
39	After go-live / AI SRE	Ongoing operations service not delivered	24/7 AI SRE monitors the system, repairs incidents, tunes cost and scaling, catches security drift and maintains runbooks	0/10	10/10
40	Integrations and automation	Integrated automation platform not delivered	Kronos Automate connects to 3,000+ services, with a dedicated automation tenant per modernized app	0/10	10/10
41	Build on the modernized system	App, agent and workflow builders not delivered	App builder, workflow builder and automation engine included for new apps, agents and workflows on the transformed code	0/10	10/10
42	Platform governance	Platform access controls, audit trails and policy enforcement not delivered	Role- and attribute-based access, audit trails and policy enforcement	0/10	10/10
43	Purpose-built models and private hosting	Purpose-built, self-hosted modernization and automation models not delivered	Kronos models purpose-built for mainframe modernization and automation, self-hostable inside the client's perimeter	0/10	10/10

	Asked-items subtotal			204/350 (58%)	350/350 (100%)
	Added platform subtotal			0/80 (0%)	80/80 (100%)
	Combined total			204/430 (47.4%)	430/430 (100%)

8. Beyond what was asked: where AI modernization breaks

The run above was one slice, one prompt, 78 files. A real estate has thousands of programs, years of data, a batch window and an audit. Modernization projects fail in the places below, and none of them can be fixed with a better prompt.

8.1 The prompt treadmill

One prompt never finishes an estate. Teams end up prompting, re-prompting and patching, and every session starts with less of the system in its head than the last one ended with.

Where it breaks	What it costs	Codegiant
Context runs out	Program 400 is converted without what was learned on program 4	Nothing depends on what a model remembers
The same input gives a different output	Rerun a program and the Java changes; nothing can be signed off twice	Same input, same output, every time
Every session decides again	Layouts, names, types and error handling drift from prompt to prompt	Decided once for the estate
Nobody can review it all	Thousands of hand-shaped programs, each a little different	Uniform by construction; nothing hand-shaped to review
The model grades its own work	The same model writes the code, the tests and the verdict	Proof is independent of the writer

8.2 Uniformity across the estate

Where it breaks	What it costs	Codegiant
One copybook, many conversions	The same record reads differently in different programs, and each passes its own tests	One answer for every program that shares it
Numbers, signs and views handled case by case	Totals, rounding and REDEFINES behave differently depending on which prompt wrote which method	One behaviour across the estate
Schema drift between waves	Wave 2 meets wave 1's tables and does not fit; rework mid-program	Every wave lands on the same design

8.3 The program outside the source

The same COBOL does different things at different sites, and much of what decides it is not in the source. A model that reads only the source is guessing, and the guess shows up as wrong totals on a busy night or a wrong status on a bad one. Sol asked about two of these facts, and none of the rest.

Where it breaks	What it costs	Codegiant
Behaviour set outside the code	Silent differences in arithmetic, signs, characters and file handling	Captured and applied per client
Source that isn't what runs	A faithful conversion of the wrong program	Caught before conversion

8.4 The data

The declarations say what a field should hold. Twenty years of production say what it actually holds.

Where it breaks	What it costs	Codegiant
Real data breaks the rules	Bad values post as zero, or the migration weekend stops at record 3 million	Every exception found and reported before cutover
Bytes no program names	Lost on the first rewrite, and nobody notices for months	Preserved
Order	SQL returns rows in a different order than VSAM did, and every control break moves	Mainframe order on every read
Generations	(0), (+1) and (-1) collapse into one table	Every generation side by side

8.5 PostgreSQL and the batch window

A correct database that misses the window is a failed migration. It also has to recover cleanly when a job dies halfway.

Where it breaks	What it costs	Codegiant
Chatty, unbatched access	Fine on a 300-record sample, hours late on production volume	Engineered for production volume
A job dies halfway	Starts again from zero, or posts the same transactions twice	Restarts where it stopped, never posts twice
Two runs at once	Two copies of a job update the same rows	Prevented, as on the mainframe
Records stored as blobs	No index, query or report without decoding bytes	Typed, queryable tables

8.6 JCL and operations

Operations runs jobs, not programs. Hand-written job scripts break the moment someone changes the JCL.

Where it breaks	What it costs	Codegiant
Jobs written by hand	Every JCL or parameter change becomes a code change and a release	Every job converted into a Java batch job that does everything its JCL did, and keeps working when the JCL changes
Step logic simplified	Steps run that should be skipped; files kept, deleted or catalogued at the wrong time	Full JCL coverage: step conditions, file dispositions, generations, parameters and in-stream data
Utility and console steps dropped	Sol dropped all six online-file close and open steps; the utility steps were hand-coded or partly ignored	The utilities run as real steps: file definition and copy, sort, delete and create, conditional return codes
Return and abend codes changed	The scheduler reacts to codes it has never seen	The scheduler sees what it sees today

8.7 When things fail

Most of a batch program is error handling, and error handling is what an AI conversion tests least. Sol's proof never triggered a single error path.

Where it breaks	What it costs	Codegiant
Error paths converted loosely or not at all	The first real I/O error in production behaves unlike the last 20 years	Same status, same message, same outcome
Invented or "fixed" behaviour	Checks, messages and abends the COBOL never had, or quirks quietly corrected	Nothing added, nothing removed

8.8 Proof that holds up

"Identical on our tests" is only as good as the tests.

Where it breaks	What it costs	Codegiant
A local compiler and happy-path inputs	The paths that hurt in production are never run	Proven against production behaviour, on the inputs that hurt
No trace, no reconciliation, no sign-off	Auditors get a chat log	Every line traceable; a certificate per program and job

8.9 Scale, cost and control

A demo on 15 programs says nothing about 6,000.

Where it breaks	What it costs	Codegiant
Cost grows with every re-prompt	Token spend and wall time climb with each retry and fix	Predictable cost per program
No impact analysis, unreadable Java	Nobody knows what a change hits, and the next team rewrites it again	Every dependency known; Java a developer can own
No path to production	A zip of code; deployment, scheduling and monitoring left to the client	Converted, proven and running on one platform

9. The Codegiant platform: where everything goes after conversion

These are the eight platform criteria scored in rows 36 to 43 of the scorecard. Sol was not asked for them, so each Sol line describes what this run delivered, not what the model could generate.

One-click deploy

OpenAI's Sol: not delivered in this run. The output stops at a zip of code, with deployment, scheduling and monitoring left to you.

Codegiant: the converted app deploys in one click to the client's choice of AWS, Google Cloud, Azure, Oracle Cloud, DigitalOcean, Cloudflare or Fly.io, or on-premises, or any Kubernetes infrastructure you run.

Database

OpenAI's Sol: a PostgreSQL schema and load scripts, with each record stored in one bytea column. Deploying it to a managed database is left to you.

Codegiant: the PostgreSQL tables go to the client's managed database, deployed with the app.

DevSecOps

OpenAI's Sol: the run hands you code, and you assemble the toolchain yourself.

Codegiant: its own toolchain with nothing to wire up: Git, CI/CD, Terraform, observability, error and performance monitoring, uptime checks, status pages, docs and code review.

After go-live

OpenAI's Sol: the run ends once the code and the proof are delivered.

Codegiant: AI SRE, 24/7. It watches the running system, repairs incidents, tunes cost and scaling, catches security drift and keeps runbooks current.

Connect

OpenAI's Sol: no integrated automation platform was delivered in this run.

Codegiant: Kronos Automate, Codegiant's own LLM, connects to 3,000+ services so you can build automations and workflows on top of your modernized transformation. Each modernized app gets its own dedicated automation tenant, running an enterprise-grade engine that supports hundreds of millions of events per month.

Build

OpenAI's Sol: none in this run.

Codegiant: new apps, agents and workflows on top of the transformed code. The app builder, the workflow builder for agents and the automation engine are all included, with no add-ons and nothing to bolt on.

Governance

OpenAI's Sol: none in this run.

Codegiant: role- and attribute-based access, audit trails and policy enforcement across everything.

Purpose Built Models

OpenAI's Sol: a general-purpose model, not one built for mainframe modernization, and no self-hosted option was delivered in this run.

Codegiant: Kronos models purpose-built for mainframe modernization and automation, which can be self-hosted inside your perimeter for added safety.

The bottom line: a model is not a modernization platform

Every row above came up in a real run: Sol's, Devin's or the Claude plugin's. None of them is visible on a 300-record sample, and all of them show up in production. A model can write COBOL-shaped Java. An estate has to be converted the same way every time, proven against the mainframe, and still run inside the window. That cannot be done with one prompt, or with a thousand.

And conversion is only the first step. Modernization means a system that can use modern technology: any cloud, real databases, automated pipelines, AI operations, and integrations with the tools your teams already use. That is the gap between migrating code and modernizing a business.

Try Codegiant's Enterprise AI Platform Today

Codegiant is an enterprise AI platform that gives you full control of the code. What comes out of the conversion is yours: no vendor runtime, no proprietary binary, no engine underneath it. You can read it, change it, and run it wherever you want.

Nothing gets bolted on afterwards. The DevSecOps toolchain, CI/CD, Terraform, observability, uptime checks and AI SRE come with the conversion, and the finished application deploys in one click to AWS, Google Cloud, Azure, Oracle Cloud, DigitalOcean, Cloudflare or Fly.io, or on-premises, or any Kubernetes infrastructure you run.

Kronos Automate, our own LLM, connects the modernized system to 3,000+ services so you can build automations and workflows on top of it. Each modernized app gets its own dedicated automation tenant, running an enterprise-grade engine that supports hundreds of millions of events per month. That is what strict modernization requirements actually demand, and it is the part most platforms stop short of: every target your own policy allows, the whole toolchain in your hands, and no binary, no runtime and no lock-in carried into the next project.

오늘 구축을 시작하세요.

엔터프라이즈급 AI와 수천 개의 커넥터를 활용해 다음에 필요한 것을 구축하세요.

시작하기 데모 요청

OpenAI's Sol vs Codegiant: Scored on COBOL Migration

Summary

What we tested

How Codegiant scores against OpenAI's Sol

OpenAI's Sol vs Codegiant: the comparison at a glance

The prompt given to GPT-6.1 Sol

1. Analysis: Sol against our deterministic facts

2. The dependency map

3. CBTRN02C to Java (asked)

4. The slice's datasets in PostgreSQL (asked)

5. POSTTRAN as a job (asked)

6. Proof of the slice (asked)

7. The complete scorecard: asked items and platform

8. Beyond what was asked: where AI modernization breaks

8.1 The prompt treadmill

8.2 Uniformity across the estate

8.3 The program outside the source

8.4 The data

8.5 PostgreSQL and the batch window

8.6 JCL and operations

8.7 When things fail

8.8 Proof that holds up

8.9 Scale, cost and control

9. The Codegiant platform: where everything goes after conversion

One-click deploy

Database

DevSecOps

After go-live

Connect

Build

Governance

Purpose Built Models

The bottom line: a model is not a modernization platform

Related Reading

Try Codegiant's Enterprise AI Platform Today

오늘 구축을 시작하세요.