Hi everyone,
Thanks to everyone who joined our webinar about unlocking COBOL unit testing using GenAI. To those who couldn't make please feel free to watch it on demand.
Webinar materials
- Webinar recording
- Webinar deck is in the attachments
- AI Unit Testing Beta Program
Quiz winners
Thank you to everyone who actively participated in the quiz as well! We’re going to share the gifts with the 5 quiz winners next week:
- Kyle W.
- Paul R.
- Jonathan S.
- Lucas M.
- Per S.
Webinar questions
As promised, below are our answers to the questions raised in our webinar - Unlock COBOL Unit Testing with GenAI, held last week. For those you that missed it, the recording can be found here.
We did our best to answers as many questions as we could with the limited time we had but below are the questions we couldn’t get to, as well as written responses to those we did.
How does the assistant mock the data for data base lookup? Can it?
In the Beta release, the testing assistant provides support to create a Program Controller and/or EXEC controller, which are the basic shell for mocking support. Any data values currently then have to be added to the unit test code manually.
The Unit Test Framework itself has a ‘snippet generator’ which can create a complete shell of every EXEC statement contained with a program,
Is the Beta testing to be done on our timeline or is there a deadline to be completed?
At this stage there is not set deadline for the Beta to be completed although obviously we would appreciate your feedback as soon as possible. If at any point we do decide on an end date, we will communicate this via this Community site.
what is the advantage over mighty tools like claude code who additionally provides giving coding inspiration and running the tests on its own?
Our VS code testing feature uses the Rocket COBOL language server to find out specific and accurate information about the COBOL program before asking the LLM to create a test artifact. Next the source editing features use VS code text api's, so the developer has the ability to "undo" the changes.
Supporting Claude code would require our testing functionality to be provides a MCP service, as well as integrating our Rocket COBOL language server.
Without the integration with the Rocket COBOL language server for COBOL information, the LLM would be left on its own to infer the information and this cause hallucinations and bad code to created.
Is Basic GitHub Copilot subscription enough? Or do we need an elevated subscription?
Whilst it is possible to run the Beta using a basic (Free) GitHub Copilot subscription, the models available in this subscription do not provide the same quality of responses as those available in the paid subscriptions. The documented Beta samples were performed using Claude Sonnet 4.5.
It is also important to note that, if you wish to try the Beta against your own code, only GitHub Copilot Business and GitHub Copilot Enterprise subscriptions do not use your code, prompts, or interaction data to train their AI models. These premium tiers ensure your intellectual property remains private and is covered by strict data protection agreements that prohibit unauthorized model training.
So, behind these command you have created skills for GitHub Copilot? Do we have access to read these skills configurations behind these .
The testing supporting does not uses "skills" but uses the vscode chat participant for the slash (/) commands.
This allows us to have deeper integration with COBOL language server to identify useful information in order to create a more accurate and directed prompt that helps create better test plans and tests.
So, unfortunately they is no skills to access or configure.
Are unit test cases injected as copybooks at runtime?
The Unit Test for sections and paragraphs are created as copybooks that are injected at compile time, resulting in a test specific executable. For tests created based on program Entry Points a separate test program is created.
Is this available in Visual Studio Professional? None of my work is done in VS Code.
At this stage, the Unit Test Assistant is only available via VS Code. However, it is important to note that the Rocket Unit Test Framework itself, is also available in both our Visual Studio and Eclipse variants of Visual COBOL and Enterprise Developer.
Most of my Visual COBOL code is interactive with events. Does this work with events?
Without understanding what type of events are being used and how they are activated, it is not possible to give a definitive answer.
However, if each event is handler in separate section/paragraph then it will be possible.
A code example will certainly help us give you better advice.
Most of my modules work on objects that come from other parts of the program. I have very few modules that take simple inputs and produce simple
Again, without concrete examples it is difficult provide a comprehensive answer, however, as a Unit within a COBOL module is defined as the smallest testable piece of code, namely a paragraph or section, then it should be possible to break down each module into numerous unit test cases using these tools.
At some point, is a connection to an Enterprise Analyzer Server ?
At this stage, there is no requirement for Enterprise Analyzer to be present to use the Testing Assistant.
In summary does it really mean your program doesn't actually run, but instead your performs become entry points where the values are past in and output passed out an asserted.
What actually runs is the code ‘under test’, which could be a paragraph, section or indeed the whole program.
How do you handle entrypoints? Stub and mocks?
If the code under test depends on a entry-point or program and is used via a CALL statement, you can choose to either ignore the statement or replace it via the use of mock controller.
we have cobol module X, calling module Y..say we have chain of calling modules, how is this handled in AI Unit testing
We define a Unit as the smallest piece of testable software. In this case, Module X or Y maybe be that but it is more likely that they will contain paragraphs and sections of code that can be tested individually. However, if the code under test from Module X contains the call to Module Y, Module Y could be mocked.
How is it different from the Enterprise Developer Eclipse Debugger feature, which provides tracing and allows us to feed values in real time for testing
Unit tests are not executed under the debugger
Next, the tests are concrete assets that can be saved, executed and reused later to ensure the validity of the code either manually by a developer or in a CI/CD environment
Do you have training video links for the Unit testing framework.
One of the pre-requisites for this Beta is the public training course - Unit Testing Framework Essentials - L440 – referenced in a previous post. This training contains videos showing the use of the Unit Testing Framework.
Videos showing the demonstration from the Webinar and a full walkthrough of the Beta Test Samples will be posted in due course, if deemed necessary.
What is the best way to introduce unit testing into a legacy application?
Ultimately this is down to resources. Obviously the quicker you build up a body of unit tests for your code, the quicker you obtain the value. That said, I would imagine for most development teams, these tests will grow organically i.e. each time you change a section of code, use the assistant to create tests for that section. If each team member follows this approach the code coverage should grow rapidly.
How do we test programs with heavy file I/O + DB calls + screen logic together?
I refer back to the definition of a Unit. One program may consist of numerous units that can be tested. If your code has a mix of business logic and ‘external’ interfaces such as the database or screen, these can be handled using the mocking support.
If you have any more questions or seek a clarification on any of the answers above then please let us know.
