How Generative AI Impacts Software Development at The Gnar

It would be difficult to make it through a day without hearing, reading, or seeing something related to Artificial Intelligence (AI). While the concept of AI is nothing new, there is a new form of AI that has generated a lot of buzz: generative AI. In the simplest terms, generative AI is a system that can generate content (text, images, code) in response to user input. This type of system has the potential to significantly impact many industries - from healthcare to legal to the software development industry.

As a consultancy, we are constantly evaluating new tools and technologies that will help us to efficiently deliver high-quality work to our clients. At this point, it's important to highlight that we are consumers of AI, not creators of it. That is to say, we won't be building the next big natural language processing tool, but we may use one to help us write a test for a certain piece of code. There are a number of AI platforms that are already proliferating in the software design and development space, and our team is happy to experiment with them.

Generative AI at The Gnar

We asked our team what tools they were using to make their process more efficient and there were two at the top of the list: GitHub Copilot and ChatGPT. But we're more than just an engineering shop - we also offer a full suite of design services and there are a number of tools that our designers use to improve the quality of their deliverables, such as MagiCopy.

GitHub Copilot

GitHub Copilot is marketed as an "AI pair programmer". It is a paywalled service ($10/mo) that was released in the fall of 2021. The tool plugs into a text editor, such as VSCode, and can make real-time code suggestions based on input from the developer.

How We Are Using It

The majority of our team is using it to increase velocity and reduce the monotony of repetitive tasks. GitHub Copilot is anticipatory so if you begin typing some code it will try to automatically complete a word, line of code, or function. This may sound simple, but when we look at time spent on the keyboard over the course of an engagement, all of those reserved keystrokes have a cost savings associated with them. As a more practical example, we are firm believers in the value of comprehensive test coverage in the applications that we build. However, writing tests is time-consuming, and by nature, there is quite a bit of repetitive setup for each file. GitHub Copilot really shines here because it can handle much of the setup and can even write complete tests for you (though we strongly caution against accepting these at face value, more on this later).

Here's what one of our team members had to say when asked how they were using AI to make their life easier:

Github Copilot! I much prefer the in-editor interface to anything prompt-based and while it often spits out nonsense for business logic it excels at test suites. [...]

Why We Are Using It

GitHub Copilot has been an easy choice for our team for a number of reasons. GitHub is a credible and familiar platform. We use it almost exclusively for storing and managing code. Moreover, GitHub was acquired by Microsoft back in 2018 so we know that they have strong financial and technical backing, which is promising for the tool's longevity. It also integrates directly with VSCode (another Microsoft brainchild) which is a favorite text editor of many of our team members and therefore has a low barrier to entry. This combined with the time and effort savings during development makes it really attractive for developers and clients alike. However, it's not all kittens and rainbows (more on this in the Challenges with Generative AI section).

ChatGPT

Chat Generative Pre-Trained Transformer (affectionately referred to as ChatGPT), is a natural language processing tool which is really just a fancy way of saying "an AI-based chatbot" which was developed by OpenAI and released in late 2022. A user can engage with ChatGPT via prompts and receive human-like responses. It is not software specific/focused - you can ask it what the meaning of life is, to build a keto meal plan for you, or to convert a Ruby method to Python. The world is your oyster.

How We Are Using It

ChatGPT is used less often than tools like GitHub Copilot but it did come up a number of times when discussing Generative AI with our team. It can be a great replacement for specific, simple questions about things like code syntax. With these types of questions, it's often faster to use a tool like this than scouring blog posts or forums like Stack Overflow. One team member finds utility in ChatGPT as a "rubber duck" when they are trying to solve a problem.

It's better curated than Stack Overflow. Sometimes AI is my "rubber duck". The process of forming a coherent question for ChatGPT helps me better understand the problem I'm working on.

Why We Are Using It

The primary reason for using ChatGPT is that it is super easy to get started. Getting started requires nothing more than an account with OpenAI and a question. Developers spend a significant amount of time learning, researching, and reading documentation. There are certain situations in which tools like ChatGPT cut down on reading so more time can be spent writing code.

MagiCopy

MagiCopy is a Figma plug-in. Figma is the choice interface design application of our design team. The plug-in can generate copy based on project type, company, desired tone, and content type.

How We Are Using It

Our design team uses MagiCopy to generate meaningful text for wireframes and mockups.

Why We Are Using It

We use MagiCopy because it provides a more realistic feel to design deliverables, which translates to a more delightful experience for our clients and their stakeholders. Without a tool like this, there would be a time investment in generating realistic copy in the specific context of a project, or worse, we'd see a lot more lorem ipsum. Additionally, as with many of the other tools mentioned in this article, MagiCopy integrates with a platform that we know and love, lowering the barrier to entry.

There's a constant struggle when designing of whether we need copy before design or design before copy. In instances when copy isn't written yet or fully approved for a website, designers will use "Lorem Ipsum" as placeholder text in their work, but it doesn't do a good job of representing how real copy will flow on a page. Not only that, but telling the brand story is important to elevating the design, so if the copy isn't final I write my own to help make a design feel more real. Having the ability to generate copy for my designs with a tool like MagiCopy I not only save time, but no longer have to default to "Lorem ipsum" when copy is not available or I'm unable to write it myself.

While these tools have the potential to enhance the design and development process, we also envision a number of broad challenges that Generative AI poses to the industry. With great power comes great responsibility…

Challenges with Generative AI

While these recent advances in Generative AI are remarkable and the conceptual utility is there, there are a number of challenges that have prevented widespread adoption at The Gnar.

• Overreliance. At this stage, the output from these tools is often flat-out wrong and still requires a careful eye to ensure what is being returned is usable and correct. Could ChatGPT build a web app for you? Probably. Would it be production-ready or usable out of the box? Almost certainly not. The responses from the tools are only as good as the level of detail and prompts that you provide to them. As users of ChatGPT and GitHub Copilot, we're still required to take everything with a grain of salt and double-check the output - it would be a mistake to become overly reliant on these tools to perform our work for us.

• Misrepresentation. This applies less to our current team but more to potential hires (and the potential hires of our clients). It is becoming increasingly easier to misrepresent a skillset using tools like this. A design candidate could use an image-based Generative AI platform to create and submit a portfolio of work just as easily as a software developer could use ChatGPT or GitHub Copilot to complete a coding assessment. While we like to assume the best, this is a serious ethical concern that will need to be monitored as these tools become more advanced.

• Intellectual Property. Generative AI tools are built and continuously trained on the data that they are fed. The more data they process, in theory, the more helpful they become. The work we produce is always considered the intellectual property of our clients. When it comes to the impact of using a plugin like Github Copilot, that intellectual property is being used as input data to improve that tool. There's a big question mark in our minds about whether companies are aware of this and more importantly, if they're okay with it.

• Impact on Cost and Quality. While we don't think AI is going to cost us our jobs or take over the world, there are specific concerns about how we work with AI that will impact software development:

In its current form, generative AI could be a repeat of the "low code" experiments of the 1990s: I think a lot of non-technical decision makers will think it can heavily automate software development. They will pay consultancies to 'use AI' to cheaply write their apps. The apps won't work very well and they need more money to fix them than it would have taken to write them properly the first time.

The takeaway from all of this? Generative AI tools are very powerful and have the potential to greatly impact the software development industry. We are still in the very early days of technology of this type and there are a lot of challenges and ethical concerns that prevent us from diving in head first. However, we plan to continue to evaluate these tools and to use them to boost productivity by assisting with repetitive tasks - but only when and where it makes sense to do so, without compromising quality.