How can you make manual testing easier?

 I found this still in the printer when I bought my early morning coffee the other day.

It reminded me of a time when I used to have 5 (yes, five) different printers on my desk (well, on a table next to it) to enable testing support for all the different and specific label printers the software I worked on had to support.

More importantly, it reminded me of the importance of making it as easy as possible to manually test things that can only (practically) be manually tested.

I assume the above output is intended to verify that the printer is working correctly.

I also assume it's used at the start of each day to verify (check) that the printer (in this case, on the automated checkout) is working correctly.

That printout is currently stuck up in front of my desk. It's a reminder to myself to regularly ask:

What could I do to make this easier to test?

Does AI mean software development will appeal to different people?

Certain personality types are attracted to software development.

The logic and absolute certainty appeals to those people.

AI removes that certainty. It's non-deterministic.

Will this put off some people who like (or need?) the absolutes?

May it attract different people with other interests and personality types?

The most important question on a PR checklist is...

Admit it, you thought I  was going to say something about Testing. Didn't you?

While testing is super important and should be part of every PR, the most important question to ask when working on something as part of a team is:

Does this change include anything you need to communicate to the rest of the team?

I've added this to PR templates and have been frustrated when "circumstances" have prevented me from doing so.

Why does this matter?

• Because if it's something low-level or fundamental, then everyone is likely to need to know it. You shouldn't rely on everyone reviewing the PR or checking everything that gets merged. 
• Because the change(s) in the PR might break the code other team members are working on.
• Because a "team" implies working together towards a common goal. Keeping secrets and not sharing things that "teammates" will benefit from knowing hurts the team and the project.

Not sharing important knowledge and information creates frustration, resentment, wasted effort, and more.

Doing things different in CI builds

Say you have a large complex solution.

Building everything takes longer than you'd like. Especially when building as part of CI checks, such as for a gated merge.

To make the CI build faster, it can be tempting to have a filtered version of the solution that only builds 'the most important parts' or filters out some things that take the most time (and theoretically change least often).

When such a filter exists, it can be tempting for developers in a team to use the filtered version to do their work and make their changes. If it makes the CI faster, it can make their development build times faster too.

However, there's an unstated trade-off happening here:

Shortening the time to build on (as part of) the CI

creates 

A reliance on individual developers to check that their changes work correctly for the whole solution.

If you get it wrong, developers (mostly?) only work with a portion of the code base, and errors can be overlooked. These errors (including things like being able to build the whole solution) can then exist in the code base for unknown periods of time......

Prompt engineering and search terms?

The prompts given to LLMs are compared to the text we can enter when searching the web.

Over time, people have learned to write search queries that will help them get better answers.

There's an argument that, over time, people will learn to write better prompts when interacting with LLMs.

However, over time, the way websites expose, format, and provide data that's fed into the search engine so that it will be more likely to be what is shown in the results for specific searches. (A crude description of SEO, I admit.)

What's the equivalent of providing data that will be used to train an LLM?

How do (or will) people make or format data to get the outcomes they want when it becomes part of the training data for the LLM?

Yes, this is like SEO for LLMs. 

LLM-Optimization? LLMO?

GPTO?

Setting up a new machine with WinGet DSC - my experience

TLDR (for Clint 😜)

• Mostly positive, would recommend
• Dev Home still crashes lots - but it is in preview, so I'll let it off
• My scripts are at github.com/mrlacey/my-config
• Many sample config scripts (from the official repo) wouldn't run (failed) for me :( 
• VS doesn't like me.

Reverse engineering a workshop

I've been working on writing a technical workshop. I've not done it before, and I couldn't find any good, simple guidelines for creating such a thing.

Having asked a few people who've delivered workshops in the past, the advice I got was very generic and more about the workshops they've proctored rather than how to structure one or put one together.

So, rather than make it up, I started by trying to reverse engineer what good workshops do.

I want the workshop to be fully self-paced and self-guided. If it can be used in group or "instructor-led" scenarios, that'll be good, too, but I don't have any plans (yet) for this.

From looking at many workshops I've completed and thinking back to those I've participated in in the past, I was struck by how many take the approach of showing a completed project and then simply listing the steps to create it. I find this approach to often be disappointing.
Yes, as a participant, I get the satisfaction of having created something but it's not something new or necessarily specific to my needs. More importantly, the reasons for each individual step weren't explained, and the reason for taking an approach when others are available (or even what the other approaches are) wasn't given. This means that I don't get the wider knowledge I likely need to be successful. Is the intention that in completing a workshop, you have the knowledge to go and build other things and the confidence to do so, having done it once before? It probably should be. 

What I find many workshops end up doing (intentionally or otherwise) is providing a series of steps to recreate a piece of software and assuming that's enough for the participants to then go off and successfully create anything else.

Yes, saying, "Anyone can follow our workshop and create X", is great. But that's not the same as a workshop that teaches reusable skills and provides the knowledge needed to go and create your own software.

I want to create a workshop as a way of teaching skills and introducing a different way of thinking about a topic.

Aside: what's the difference between a workshop and a tutorial? I think it's that workshops are longer. Possibly a workshop is made up of a series of tutorials.

After initially struggling, I eventually concluded that a workshop is like teaching anything else. With clear learning goals and a structure, it's a lot easier to plan and create.

In this way, writing the workshop was a lot like writing a book. Only without an editor chasing me for progress ;)

More thoughts on this topic another day. Maybe.
Although, it has got me thinking about what I'll write next...

If you're interested in how my efforts turned out, you can see the results of them here.

If you only write one document

If you only write one document (as part of developing software), make it the test plan.

Do it before you start writing the code.

This applies to new features and bug fixes.

• This also becomes the spec.
• It allows you to know when you've written the code to make do everything you should.
• It will make working as part of a team easier.
• It will make code reviews faster and easier.
• It will make testing faster.
• It will make creating automated tests easier. (You can even write them before the code-TDD style.)
• It will make things easier when the customer/client/boss changes their mind or the requirements.
• It will make future maintenance and changes faster.
• It will make creating complete and accurate documentation easier.
• You are more likely to be creating technical debt that you're unaware of without this.

If you don't:

• There will be lots of guessing.
• There will be lots of unvalidated assumptions.
• There will be lots of repetition of effort. (Working out what needs to be done and how to do it.)
• More effort will be wasted on things that aren't important.
• Code reviews and testing of the code will be slower and involve more discussion and clarification.
• You are more likely to ship with bugs.
• Future changes (big fixes or adding new features) will be slower.
• Future changes are more likely to introduce other bugs or regressions.

Who said it doesn't matter

That someone admitted [bad practice the business would not like to admit to] is not the issue.

The problem isn't blaming who said it.

The problem is that it is the culture. 

Trying to hide the issue or blaming someone for admitting it doesn't help. It encourages bad practice, which really only makes things worse.

 

Detecting relevant content of interest

When AI is actually a bit dumb. 

Consider something I'm seeing a lot:

• Content (e.g. news) app shows lots of different content.
• You read an article in the app within a specific category.
• Several hours later, an automated backend process tries to prompt re-engagement.
• The backend process looks at what categories of content you've looked at recently.
• It notices a recent article that is in that category and is getting lots of views.
• The backend process then sends a notification about that content as you're likely to be interested. (It knows you read content in the category, and lots of people are reading this new article. It should be of interest.)
• But the "assumption" was based on a lack of consideration for articles already read. (It was of interest, that's why you read it several hours ago.)
• Enough people click on these notifications to make them look like they're doing a good job of promoting re-engagement.
• People click on the notifications because they might be related to or following on from what they read earlier and not realizing that it is the exact same article.
• Analytics only tracks openings from the notifications, not how much time is spent reading the article they were using to promote re-engagement.

Analysis of analytics doesn't flag this, and the opacity of "the algorithm" doesn't make it clear this is what's happening.

All the while, many people get wise to these pointless notifications and turn them off, and so many miss something actually useful in the future.

I know there are more complex and varied scenarios than mentioned above, including how the above could be considered a successful "engagement" in other ways.

The main thing I take away from this is that I don't want to be creating software that tries to do something clever and useful without being able to accurately tell it is being successful and providing value.

Creating something that is trying to be clever but comes across as stupid because it doesn't make wise or appropriate use of the information it already has does not excite or interest me.

Software development = basketball

This might be a leap but go with me for a minute.

"Lessons in basketball are lessons in life"

 

It's a cliched phrase that was drilled into me at basketball training camps and through basketball-related movies when I was young. We weren't just being encouraged (forced?) to learn lessons that would help us play better basketball, these lessons would help throughout our lives.

Thinking today about the importance of fundamentals, I wonder if the world would be a better place if more developers (had) played basketball.

Sticktoitiveness and software development

I recently heard that there is a common character trait among many developers in that they won't stop working on a problem until they've solved it.

I've always identified as having a similar but different trait, I won't give up when trying to solve a problem.

I came to this trait as a result of some of my first jobs in the industry. Due to the internet and world being as they were, and in combination with the companies, teams, and projects I was working on/with, there was no option to say "I don't know how" and give up. The problem needed to be solved, there was no one to ask who might know, and so I had to figure it out. That's what I was there for. That's what I was paid for.

Did this bug report get me banned from Visual Studio?

 As an avid user of Visual Studio and a developer of many Visual Studio extensions, I have a strong interest in enhancing the discoverability and user-friendliness of extensions. I was pleased to learn about the recent implementation of a requested feature and eagerly went to explore it.

Recently, I've also been exploring the use of WinGet DSC to configure a new laptop and have been experimenting with .vsconfig files to streamline the process.

During these investigations, I encountered an issue regarding the use of extensions containing "Extension Packs" (references to other extensions that should also be installed). Unfortunately, attempting to include them resulted in installation failures without any accompanying explanation for this limitation. Through a process of elimination, I confirmed that the inclusion of extension packs was the cause.

I submitted a bug report detailing my findings, which can be found [link to the original report, which was unfortunately removed]. Regrettably, I discovered that my access to the site has since been restricted, citing violations of the Community Code of Conduct. 

Upon revisiting my initial post, I can only speculate that my direct and passionate writing style may have been misunderstood as impolite or disrespectful, but am unsure if this is the issue. I acknowledge the importance of maintaining politeness and respect in online interactions and am committed to improving in this regard.

I am left wondering if utilizing AI to refine my expressions to ensure a consistently polite and respectful tone may be a helpful approach moving forward. Perhaps this precautionary measure could prevent unintentional misinterpretations. 

Below is what I posted.

I share it here as an example (and warning?) to others. Be polite and respectful!

Themes of DDD North 2024

This last weekend, I was excited to get to speak at the DDD North conference again.

As a one-day, five-track conference there was a lot going on and a lot of varied content.

Of the sessions I attended and the discussions I had with other attendees, I noticed lots of mentions of:

• AI
• Testing (in a positive light)
• General good development practices rather than talk about specific tools or technologies.

Yes, I recognise that the talk I gave about the importance of documentation and testing as we use more AI tooling while developing software likely skewed my thinking and what I was more inclined to notice. It was just nice to not be the only person saying positive things about testing software. (Although at least two speakers did make jokes about writing tests so there's still a long way to go.)

The increased focus on generally applicable "good" practices was also good to see. While learning about a new framework or technology is useful in the short-terms or for specific tasks, spending time on things that will be valuable whatever the future holds feels like a better use of time.

While I'm still waiting for the official feedback from my talk (sorry, no video) upon reflection, I'm glad I did it and it was a good thing for me to do. I don't want to give a talk that anyone could give and so basing it on my experiences (& stories) is good rather than reading official descriptions of technologies, describing APIs, or showing trivial demos. I also want to do in-person events in ways that benefit from being "in person". This talk wouldn't have worked the same way as a recording and I wouldn't have got as much from it either. If I could just record myself talking about the subject and released it as audio or a video I'd have done that but it wouldn't be the same or as good. Although, it might have been less work. Maybe I'll do that in the future though.

 Here's me during the talk in front of a perfectly timed slide ;)

Lack of nuance

 No nuance is almost always incorrect!

Yes, "almost" is very important in that statement.

If you get a response/answer/instruction without any acknowledgement of the nuances, you're almost certainly not getting the full picture. 

How do you know the importance of what is missing, if you don't know what's missing?

Always use a regex rather than string manipulation

 Although, maybe not always:

It's almost as if absolute claims aren't always correct.

The required amount of documentation and testing will vary

The amount of documentation and the number of tests required will vary between:

• code bases
• the stage of development
• the people working on it
• what the code does
• more....