The field of artificial intelligence (AI) is rapidly evolving, and large language models (LLMs) are playing a critical role in the development and deployment of AI-powered applications. LLMs are a type of AI that are trained on massive datasets of text and code. They can be used for a variety of tasks, including natural language processing, machine translation, and text generation. Since the possibilities are almost endless as to what LLM’s can do, there are a few core angles to take a look at how to slice this.
- Industry / Domain Knowledge: The more you know about the domain or industry the better you will be able to clarify the problem for GPT or any other LLM to be able to solve the task for you. Since LLM’s have a large vocabulary, you should bring your own expertise to the table.
- Prompt Engineering / Communication Skills: Prompt engineering comes down to the ability to communicate what you want the machine to do. Sure there are tricks that people have and tons of prompt templates and plugins out there. If you know how to write, you can GPT to do just about anything for you.
- Automation Thinking: Several years ago, I’d have said you needed to know how to program to automate processes. These days, automations can be done in NoCode or Low Code platforms, so all you need to know is how to break down the problem into discreet steps.
Unlike traditional AI, where you need to develop a strong foundation in computer science and mathematics, you don’t need to go that deep with today’s toolkits. If you spend some time to learn a tool like Make, Zapier, Zoho Flow, or Microsoft Power Apps, you can integrate GPT/LLM in your workflow as long as you pick some API basics.
There are a few different paths you can take, and as I mentioned before you don’t need to have a programming background, just some industry knowledge and good written communication skills.
Level 1: Get Started with ChatGPT/GPT Powered Apps
- Start out with ChatGPT, Bing Chat, Bard, or Poe.
- Learn about the different ChatGPT/GPT powered apps that are available.
- Choose an app that interests you and start using it. It could be one like Everyprompt, Promptable, Brancher or one like Jasper or Rytr.
- Experiment with the different features of the app and see what you can do with it. You can look into built in automations or plugins
- Read the documentation for the app and learn as much as you can about how it works.
Level 2: Become a Prompt Engineer on ChatGPT/GPT
- Learn about the different ways to write prompts for ChatGPT/GPT. There are tons of sites like ShareGPT that have great examples from others. Poe users also share their conversations.
- Experiment with different prompts and try giving it better and better contexts, examples, and your own information to see how to get it exactly what you want.
- Join different Discord groups where people discuss how to get better at prompt engineering.
- Start writing your own prompts, share them and see if they are useful for others.
Level 3: Use GPT API with NoCode Automation, App Builders
- Learn about the different NoCode app builders that are available. A good starting point is Bubble, or Softr ( with Airtable or Google Spreadsheets)
- Choose a NoCode app builder that interests you and start using it. Bubble allows you to connect with APIs and Airtable. Start with that.
- Experiment with the different features of the NoCode app builder and see what you can do with it. Bubble can pull data from Airtable, send it to GPT API, and then save it back to Airtable.
- Continue learning how to use other NoCode tools that augment Bubble including Bubble Plugins, or those that connect with the Database you end up using like Airtable
Level 4: Create Workflows to Automate Tasks with NoCode
- Learn about the different NoCode automation that are available. A good starting point is Zapier, Make with Google Spreadsheets and Airtable
- Choose a NoCode automation with a database that interests you and start using it. I would recommend definitely using Airtable with Make. It’s a power couple.
- Experiment with the different features of the NoCode automation tool and see what you can do with it. You can create a simple app with Airtable Forms and then use Make to take the data you save and create other information in another Table.
- Continue learning how to use other NoCode tools like Bubble or Softr to create better looking user interfaces on top of the data in Airtable.
Level 5: Use GPT API with Code, make your own APIs
- Pick up Python or TypeScript/Javascript since those are the easiest to get start with to make APIs. If you know Java, Scala, C# great, go to town.
- Choose a programming language that you are comfortable with and start using it. I strongly recommend Python or Typescript. You’ll see later why.
- Experiment with the different features of the GPT API and see what you can do with it. You can even ask ChatGPT to show you how to use GPT API with the programming language of your choice.
- Read the documentation for the GPT API and learn as much as you can about how it works.
Level 6: Create Workflows to Automate Tasks with Code
- Use the programming language of your choice to create workflows that automate tasks. You can do this with a programming language that reads and writes from a database.
- Experiment with different workflows and see how they can save you time and effort.
- Share your workflows with others and help them automate their tasks.
- My first attempt was with Python Code + GPT API + Airtable. It just seemed to work well, and ChatGPT showed me how.
Level 7: Use GPT API with your Data / a Framework
- Learn about the different ways to use the GPT API with your own data or a framework. LlamaIndex and LangChain are the best ones I’ve used in Python. There are others like Window.AI and LangChain.js use JavaScript/TypeScript.
- Experiment with different ways to use the GPT API + LangChain + LlamaIndex and see what you can do with it. I made a chatbot that talked to the internet with SerpAPI and connected it to my own API backed by my own database.
- Read the documentation for LangChain, LlamaIndex and learn as much as you can about how it works. Get on their Discord Servers (LlamaIndex Discord , LangChain Discord) to learn from others.
Level 8: Use GPT API with your Data / a Framework to Make your own APIs
- Use the LangChain / LlamaIndex to create your own APIs that can be used by others.
- Experiment with different ways to ingest data and create LLM powered APIs and see how they can be used to solve real-world problems.
- Share your APIs with others and help them solve their problems.
Level 9: Create Workflows to Automate Tasks with your Data /a Framework
- Use the LangChain/ LLAMA Index with your data or a framework to create workflows that automate tasks. Could consider using something like Airflow or DAGster to run long running complicated workflows.
- Experiment with different vector databases, different chunking lengths, and different embedding methods and see how they can improve your workflows.
- Share your workflows with others and help them automate their tasks.
Level 10: Use Another LLM other than GPT
- Learn about other LLMs that are available. Cohere… LLAMA (from Meta), Alpaca, Vicuna
- Choose an LLM that interests you and start using it. Most can be used with Python, you can also use LangChain to switch between different ones.
- Experiment with the different features of the LLM and see what you can do with it. You may find some LLMs are better than others. The ones that have billions of dollars behind them tend to be really good, but a few get pretty close for being trained with $500 dollars.
- Read the documentation for the LLM and learn as much as you can about how it works.
Level 11: Fine tune / use your own models
- Learn about how to fine tune models or use starting points like Alpaca, Vicuna, GPT4all
- Experiment with different fine-tuned models, and see if they are good as the commercially available APIs like GPT or Cohere
- Use your model to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.
Level 12: Build your own LLM Model with your own Training Dataset
- Learn about how to build your own LLM model.
- Choose a framework that you are comfortable with and start building your model.
- Experiment with different parameters and see how they affect the output of your model.
- Train your model on a dataset of your choice.
- Use your model to generate text, translate languages, write different kinds of creative content
This is just a roadmap, and you may not need to follow it exactly. However, it should give you a good starting point for your journey to becoming an LLM/AI engineer.
No matter which path you choose, you will need to stay up-to-date on the latest trends in AI. This means watching Youtube Videos, participating on Discord groups, and experimenting on your own. Unfortunately this industry is so new, the only way to get it is to do it. It is also important to network with other AI professionals. The more people you know that know more than you, the better you’ll get.
If you are passionate about AI and have the skills and knowledge to succeed, then a career as an LLM/AI engineer is a great option. With a growing demand for AI-powered solutions, there are many opportunities for qualified engineers.