Showing posts with label unittest. Show all posts
Showing posts with label unittest. Show all posts

I Co-Wrote 88 Unit Tests Using AI: A Developer's Journey

Labels: , , , , , , ,

Testing has always been one of those tasks that developers know is essential but often find tedious. When I decided to add comprehensive unit tests to my NoteBookmark project, I thought: why not make this an experiment in AI-assisted development? What followed was a fascinating 4-hour journey that resulted in 88 unit tests, a complete CI/CD pipeline, and some valuable insights about working with AI coding assistants.

(Version française ici)

The Project: NoteBookmark

NoteBookmark is a .NET application built with C# that helps users manage and organize their reading notes and bookmarks. The project includes an API, a Blazor frontend, and uses Azure services for storage. You can check out the complete project on GitHub.

The Challenge: Starting from Zero

I'll be honest - it had been a while since I'd written comprehensive unit tests. Rather than diving in myself, I decided to see how different AI models would approach this task. My initial request was deliberately vague: "add a test project" without any other specifications.

Looking back, I realize I should have been more specific about which parts of the code I wanted covered. This would have made the review process easier and given me better control over the scope. But sometimes, the best learning comes from letting the AI surprise you.

The Great AI Model Comparison



GPT-4.1: Competent but Quiet

GPT-4.1 delivered decent results, but the experience felt somewhat mechanical. The code it generated was functional, but I found myself wanting more context. The explanations were minimal, and I often had to ask follow-up questions to understand the reasoning behind certain test approaches.

Gemini: The False Start

My experience with Gemini was... strange. Perhaps it was a glitch or an off day, but most of what was generated simply didn't work. I didn't persist with this model for long, as debugging AI-generated code that fundamentally doesn't function defeats the purpose of the exercise. Note that at the time of this writing, Gemini was still in preview, so I expect it to improve over time.

Claude Sonnet: The Clear Winner

This is where the magic happened. Claude Sonnet became my co-pilot of choice for this project. What set it apart wasn't just the quality of the code (though that was excellent), but the quality of the conversation. It felt like having a thoughtful colleague thinking out loud with me.

The explanations were clear and educational. When Claude suggested a particular testing approach, it would explain why. When it encountered a complex scenario, it would walk through its reasoning. I tried different versions of Claude Sonnet but didn't notice significant differences in results - they were all consistently good.

The Development Process: A 4-Hour Journey


Hour 1-2: Getting to Compilation

The first iteration couldn't compile. This wasn't surprising given the complexity of the codebase and the vague initial request. But here's where the AI collaboration really shined. Instead of manually debugging everything myself, I worked with Copilot to identify and fix issues iteratively.

We went through several rounds of:

  1. Identify compilation errors
  2. Discuss the best approach to fix them
  3. Let the AI implement the fixes
  4. Review and refine

After about 2 hours, we had a test project with 88 unit tests that compiled successfully. The AI had chosen xUnit as the testing framework, which I was happy with - it's a solid choice that I might not have picked myself if I was rusty on the current .NET testing landscape.

Hour 2.5-3.5: Making Tests Pass

Getting the tests to compile was one thing; getting them to pass was another challenge entirely. This phase taught me a lot about both my codebase and xUnit features I wasn't familiar with.

I relied heavily on the /explain feature during this phase. When tests failed, I'd ask Claude to explain what was happening and why. This was invaluable for understanding not just the immediate fix, but the underlying testing concepts.

One of those moment was learning about [InlineData(true)] and other xUnit data attributes. These weren't features I was familiar with, and having them explained in context made them immediately useful.


InlineData in the code


Hour 3.5-4: Structure and Style

Once all tests were passing, I spent time ensuring I understood each test and requesting structural changes to match my preferences. This phase was crucial for taking ownership of the code. Just because AI wrote it doesn't mean it should remain a black box. Let's repeat this: Understanding the code is essential; just because AI wrote it doesn't mean it's good.

Beyond Testing: CI/CD Integration

With the tests complete, I asked Copilot to create a GitHub Actions workflow to run tests on every push to main and v-next branches, plus PR reviews. Initially it started modifiying my existing workflow that takess care of the Azure deployment. I wanted a separate workflow for testing, so I interrupted (that's nice I wasn't "forced" to wait), and asked it to create a new one instead. The result was the running-unit-tests.yml workflow that worked perfectly on the first try.

This was genuinely surprising. CI/CD configurations often require tweaking, but the generated workflow handled:

  • Multi-version .NET setup
  • Dependency restoration
  • Building and testing
  • Test result reporting
  • Code coverage analysis
  • Artifact uploading

Code coverage


The PR Enhancement Adventure

Here's where things got interesting. When I asked Copilot to enhance the workflow to show test results in PRs, it started adding components, then paused and asked if it could delete the current version and start from scratch.

I said yes, and I'm glad I did. The rebuilt version created beautiful PR comments showing:

  • Test results summary
  • Code coverage reports (which I didn't ask for but appreciated)
  • Detailed breakdowns.

PR display


The Finishing Touches

No project is complete without proper status indicators. I added a test status badge to the README, giving anyone visiting the repository immediate visibility into the project's health.

test status badge


Key Takeaways


What Worked Well

  1. AI as a Learning Partner: Having Copilot explain testing concepts and xUnit features was like having a patient teacher
  2. Iterative Refinement: The back-and-forth process felt natural and productive
  3. Comprehensive Solutions: The AI didn't just write tests; it created a complete testing infrastructure
  4. Quality Over Speed: While it took 4 hours, the result was thorough and well-structured

What I'd Do Differently

  1. Be More Specific Initially: Starting with clearer scope would have streamlined the process
  2. Set Testing Priorities: Identifying critical paths first would have been valuable
  3. Plan for Visual Test Reports: Thinking about test result visualization from the start

Lessons About AI Collaboration

  • Model Choice Matters: The difference between AI models was significant
  • Conversation Quality Matters: Clear explanations make the collaboration more valuable
  • Trust but Verify: Understanding every piece of generated code is crucial
  • Embrace Iteration: The best results come from multiple refinement cycles

The Bigger Picture

This experiment reinforced my belief that AI coding assistants are most powerful when they're true collaborators rather than code generators. The value wasn't just in the 88 tests that were written, but in the learning that happened along the way.

For developers hesitant about AI assistance in testing: this isn't about replacing your testing skills, it's about augmenting them. The AI handles the boilerplate and suggests patterns, but you bring the domain knowledge and quality judgment.

Conclusion

Would I do this again? Absolutely. The combination of comprehensive test coverage, learning opportunities, and time efficiency made this a clear win. The 4 hours invested created not just tests, but a complete testing infrastructure that will pay dividends throughout the project's lifecycle.

If you're considering AI-assisted testing for your own projects, my advice is simple: start the conversation, be prepared to iterate, and don't be afraid to ask "why" at every step. The goal isn't just working code - it's understanding and owning that code.

The complete test suite and CI/CD pipeline are available in the NoteBookmark repository if you want to see the results of this AI collaboration in action.


Reading Notes #561

Labels: , , , , , , , , , , , , , , , , ,
It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

The suggestion of the week

Programming

Low Code

Miscellaneous

~Frank

Reading Notes #525

Labels: , , , , , , , , , , , , , ,

Picture of the bird CraneUsually, it's on Monday, I know, but a technical issue prevented me to publish faster this week... 

You think you may have interesting content, share it!

Cloud

Programming

Podcast

Miscellaneous


~Frank


Reading Notes #496

Labels: , , , , , , , , ,
Good Monday, 

time to share my reading notes. Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed. 

You think you may have interesting content, share it!

Cloud

Programming

Podcasts

  • Make money with open source software (Software Engineering Unlocked) - Open source software is another world: Unknow, different, scary. This week guest shares his experience and story, while transitioning into Open-Source.

~Frank


Reading Notes #428

Labels: , , , , , , , , , , , , , , , , , ,

Frank in Kayak smilling at the camera
Every Monday, I share my "reading notes". Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting.

It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!

Cloud


Programming


Miscellaneous

  • APIs in the 2020s Panel (.NET Rocks!) - A virtual panel of awesome speakers that talked about API, REST, GraphQL, oData and so more. Lovely episode.
  • 471: How to Say No Without Saying No, with Lois Frankel (Coaching for Leaders) - Saying No... being open. Really interesting topic. I put Lois Frankel: Nice Girls Don’t Speak Up or Stand Out in my to read list, that book maybe written for women in mind, but I think it is really interesting.
  • Leveraging Our Emotional Goals (Developer Tea) - An interesting episode that talk about goals and what we need to do (or not) to achieved them.
  • 203: Updating Open Source Projects (https://castbox.fm/episode/203%3A-Updating-Open-Source-Projects-id2117504-id267802246) - As I just create my first version in one of my open-source project I found the topic more than interesting... Thank you, great show.

Miscellaneous


Books

Cover of the Book The Impossible first.
  The Impossible First: From Fire to Ice—Crossing Antarctica Alone

  Author: Colin O'Brady

  An incredible adventure yes around the globe, but more important over the personal limits. I found this book very inspiring. I felt following him across the Antarctica... In the blizzard or in those deep moment. Great memoir, great adventure.





~enjoy!

 

Reading Notes #409

Labels: , , , , , , , , , , , , , , , ,

Every Monday, I share my reading notes. Those are the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

Cloud

Programming

  • CLI vs GitKraken Git GUI Speed Test (Jesus Murillo) - I love GitKraken, I still use the CLI and the integrated git in VSCode, but to really see things... It's the best, and wow it looks cool.

Podcasts

  • #23: John Sonmez - Becoming a Finisher, Part 1 (The Solo Coder Podcast) - I know, read and follow Simpler Programmer. It was so great to heard all the work that was put in to arrive to the current status. So many of us think it's a night success... Looking forward to listening to part 2.

Miscellaneous



Reading Notes #337

Labels: , , , , , , , , , , , , , , ,
IMG_20180707_220101

Cloud



Programming



Data



Miscellaneous




Reading Notes #334

Labels: , , , , , , , , , , , , , , , , , ,
canadaflag

Suggestion of the week

  • HTTPS Is Easy! (Troy Hunt) - A wonderful series of 4 videos that explains how to get secure with https. A must!

Cloud


Programming


Miscellaneous


Reading Notes #304

Labels: , , , , , , , , , , , , , , , , , , ,
IMG_20171108_160315

Cloud


Programming


Databases


Podcast


Miscellaneous



Reading Notes #283

Labels: , , , , , , , , , , , , , , , ,

June9Suggestion of the week


Cloud


Programming


Databases


Miscellaneous


Reading Notes #166

Labels: , , , , , , , , , , , , , , , , , , ,

IMG_20141123_093027577_HDRSuggestion of the week


Cloud


Programming


Miscellaneous


~Frank B.


Reading Notes #164

Labels: , , , , , , , , , , ,

happy-movember-magnet

Suggestion of the week


Cloud


Programming

 

Miscellaneous

~Frank B

Reading Notes #155

Labels: , , , , , , , , , , , , , , , , , , ,

CloudenFrancais_cover_400Suggestion of the week


Cloud


Programming

  • Git: It’s Just Data! - This post shows us Git from a database point of view. Julie also gives really good reference for tool and book.

Databes


Miscellaneous


~Frank


How to be sure that your PowerShell script are doing what you think?

Labels: , , , , , ,
Recently, I was required to write PowerShell script to automate some Microsoft Azure deployments. I was at first really happy to share those scripts because I knew other developers would use it. They would add functionalities and the library would grow. The idea was nice, but how to be sure that while adding some functionalities they don’t break something else? I didn’t know any unit test framework for Powershell, so I decided to do a quick search online. I was sure some kind of framework already exists. What I didn’t expect, however, was to find a framework that was really easy to use, complete and free! In this post, I will introduce you to Pester, a wonderful PowerShell framework available on github.

What is Pester?

Pester is a Business Development Driven (BDD) unit tests framework that implements a lot of functionalities like mock and exceptions management.

How to Install

The framework is available on github.com but it can also be downloaded from different repositories. My personal favorite is to download it from Chocolatey because it’s the most painless method. The Chocolatey’s command to install Pester is:
cinst pester

Before continuing, if you try a Pester command right now, you will probably have the error message: “The term ‘___’ is not recognized as the name of a cmdlet…”. One way to fix that is to add Pester to your Profile. This can be done by executing the following command.

{Import-Module "C:\Chocolatey\lib\pester.2.0.4\tools\Pester.psm1"} | New-Item -path $profile -type file -force;. $profile

Get Started

Now that Pester is available on your machine, let’s start by creating our first test. Pester can scaffold for you an empty script file and a test file. If you add Pester to an already existing library, don’t be scared, Pester will not override the existing files.

Open a PowerShell console and type the following command:

New-Fixture c:\dev\pesterDemo pesterDemocd c:\dev\pesterDemo

You can try the new test by invoking Pester with this command:

Invoke-Pester

ResultDefaultPesterTest

As you can see a test already exists, and it failed.

Simple test

Let’s create a reel test that will read an XML file to extract a property. Here is the content of the XML file and the code for both the script and the test file.

manifest.xml
<service>
    <name>Employee Provider</name>
    <version>1.2</version>
</service>

pesterDemo.Tests.ps1
$here = Split-Path -Parent $MyInvocation.MyCommand.Path
$sut = (Split-Path -Leaf $MyInvocation.MyCommand.Path).Replace(".Tests.", ".")
. "$here\$sut"

Describe "#pesterDemo#" {

    Context "Test with reel XML document."{

        It "Loads a reel manifest." {
            $xml = getManifest ".\manifest.xml"
                $xml.OuterXMl | Should Be "<service><name>Employee Provider</name><version>1.2</version></service>"
        }

        It "Return the service name."{
            $sName = GetServiceName ".\manifest.xml"
                $sName | Should Be "Employee Provider"
        }

    }
}

pesterDemo.ps1
function GetManifest($manifestPath){
        if(Test-Path $manifestPath)
        {
            [xml]$xml = Get-Content $manifestPath
            return $xml
    }
    else{
        Throw "Error. Ïnvalid Manifest file path."
    }
}

function GetServiceName($manifestPath){
        $manifest = GetManifest($manifestPath)
        return $manifest.service.name
}

Result of the test

The script file contains two simple functions. The first one [GetManifest] versifies if the file exists and returns his content. Otherwise, an exception is thrown. The second function [GetServiceName] retreives the content of the XML file, and returns the value of the property [name] of the node [service].

The test file contains two tests. The first one Loads a reel manifest test that the manifest.xml file is correctly loaded and validates the content. The second test Return the service name, validate that the name of the service is the same as expected. We can now invoke the tests.

Invoke-Pester

ResultFirstPesterTest

As you can see, all tests succeeded and the output is really clear. Now what if the path passed in parameter doesn’t exist? Let assume that in our design, we wanted that the code to throw an exception. A good thing for us, Pester already has everything to manage exceptions.

Let’s do a test to validate this “requirement.” First, define a code block that calls [GetManifest] with a wrong path. Then pipe the result in the Pester command Should. If you add the filling code to the pesterDemo.tests.ps1, and invoke-Pester all tests should succeed.

It "Throw an exception when loads a manifest with invalid path." {
{$xml = GetManifest ".\WrongPath\manifest.xml" }  |  Should Throw
}

Test with Mock

Sometimes, we don’t want our tests to interact with reel components. That’s why mocks are so useful. Thankfully for us, creating a mock with Pester is a piece of cake. In all the previous tests, a real XML file was used. To make some validation on the content of the file, I could have as many different files as different scenarios, or simpler I could use some mock.

With Pester, we must set up a context where our mock will be present. Here is a test where I mock the XML file. As expected, any path will work since the file is not reel.

Context "Test with Mocks."{

    Mock GetManifest{return [xml]"<service><name>Employee Provider</name><version>1.2</version></service>"}

    It "Any path will works"{
        $xml =  GetManifest -MockWith ".\wrongPath\manifest.xml"
            $xml.OuterXMl | Should Be "<service><name>Employee Provider</name><version>1.2</version></service>"
    }
}

Wrapping up

Of course, this little introduction is not an exhaustive list of all the features of Pester. I hope that with these little code snippets, I gave you the drive you needed to test your PowerShell scripts. So next time you write this amazing script to deploy your app on Microsoft Azure, think of Pester.

References



~Frank


Reading Notes #132

Labels: , , , , , , , , , , , , , , , , , , ,

TypeScript_CoverSuggestion of the week

  • TypeScript for C# Programmers (Steve Fenton) - Great book that in less than hundred pages, explains to me how to code in TypeScript. I feel so comfortable already I will add TypeScript in my next web project, and I will strongly recommend this book to everyone. If you are a C# developer, know your base in JavaScript this book is available in PDF for free!

Cloud


Programming


Miscellaneous



Reading Notes #102

Labels: , , , , , , , , , , , , , , ,
trafficLightSuggestion of the week
Cloud
Databases
Programming
UX
Integration
Miscellaneous
~Frank






Subscribe to: Posts (Atom)