When it comes to AI, C++ is in a unique position. Unlike most modern languages, it’s compiled directly to assembly code, and doesn’t run under the confines of a runtime system. This means it’s highly optimized and extremely fast. But the tradeoff is that it’s a much more difficult language to use, as you have to spend more time carefully freeing up memory as you go.
In other words, C++ has a steep learning curve and can be a lot of work to use effectively in the context of AI.
But with that comes advantages. Because C++ lets you work so close to the processor, it’s the chosen language for many of the foundational AI libraries used by other languages such as Python. And if you master C++ and AI, you could help take part in building such libraries.
C++ also has a huge advantage over other languages in one vital aspect of AI: With the help of a framework called CUDA, you can compile and execute your C++ code directly on NVIDIA GPUs. Remember, GPUs are vital to AI because they have thousands of cores, unlike typical CPUs that might have up to 16 or 20 cores.
This means there are a couple layers to learning how to utilize C++ in the service of building out AI apps and services. Number one is simply making use of existing libraries to add AI functionality to your applications. Second is drilling down into the libraries themselves. Before we dive into those, however, let’s talk about how you can learn the foundations of AI.
Learning the Foundations of AI
AI covers many different technologies. Here are some starting points:
- Study machine learning and deep learning concepts. There are many core concepts here, starting with supervised learning, unsupervised learning, and reinforcement learning.
- Delve a bit more into deep learning: This is a specialized form of machine learning. It’s the type of learning used by today’s generative AI, such as what ChatGPT uses.
- Learn the types of models. You’ve probably heard the term Large Language Model (LLM). This is a type of model used by AI for processing natural human language. ChatGPT, for example, uses an LLM to process language that you supply to it through prompts, and then provide an answer that sounds like natural language. But there are many other types of models; learn about sentence transformer models, as these are closely related to LLMs, then look into models that specialize in images and pattern recognition.
- Learn about neural networks: Study some basic concepts of neural networks, because these are how the models are stored. You don’t need to go very deep here; just grasping the high-level ideas is often enough.
Pro tip: At this stage, you’ll be using libraries that accomplish the work of the above concepts. That means you don’t need to know how to write code that performs supervised learning, for example; instead, you’ll be calling libraries that do that for you.
Also, ChatGPT knows a great deal about itself and how it works. You can start there with prompts such as, “I’m new to machine learning. What core concept should I learn?”
Adding AI to your C++ Applications
In order to add AI to your C++ applications, you’ll need to become familiar with the following libraries. That’s after you master certain concepts of C++, especially how templates work, how to use smart pointers, and one of the newest additions to the language, lambda expressions.
Now here are some libraries you’ll want to learn; the first two are the most important:
- TensorFlow C++: TensorFlow is one of the most used AI libraries today. It was originally built by Google. It’s written mostly in C++, but uses a lot of Python as well. It’s primarily intended to be used by Python programmers; however, it has a complete set of C++ bindings, giving C++ coders full access to it.
- PyTorch C++: PyTorch is a bit older than TensorFlow and is used a lot in research and academia. Originally a machine learning library, it’s been extended to provide full support for modern AI including natural language processing.
- ONNX: This is an open-source ecosystem with the main goal of standardizing neural networks and AI models. You might need to use the C++ bindings if you plan to load different types of models into your app. (However, TensorFlow and PyTorch both use ONNX under the hood. Still, it’s important to learn how to use it in case you need to dig deeper.)
- OpenCV: The Open Source Computer Vision library is a great example of where C++ really shines in AI. The original version is quite old, having been released in 2000. However, it has evolved over the years, and is still maintained today. It’s considered one of the most important computer vision libraries today. It’s written in C++.
- ML Pack: This is a machine learning library written mostly in C++. Like OpenCV, it’s been around a while, since 2008. But similarly, it’s continued to evolve. It has a large number of machine learning algorithms, and is a great place to learn how to use them.
- Shark: This is another machine learning library. It’s also written in C++. It’s about the same age as ML Pack, and is worth learning.
Pro tip: Although C++ can certainly handle data analysis, unfortunately there aren’t a lot of higher level libraries available for crunching massive amounts of data. Much data analysis, data science, and data visualization is typically handled through Python. If you want to do add any kind of data science to your resume, you’ll probably want to embrace Python for that part.
Contributing to the C++ AI Libraries
Before we dive into this one, let’s be blunt: This is some seriously advanced stuff we’re talking about here. In addition to becoming an expert in C++, you’ll need to become an expert in how AI works—not just using libraries, but the actual mathematics and foundations of AI. This means we can’t fit everything here, but we can point you in the direction to get started.
To work at this level of AI, you need to take things sequentially. And that means starting with the mathematical foundations.
The math behind AI requires you master three general topics:
- Linear Algebra: This branch of math deals with vectors and matrices and the operations behind them. In the same way you can add and multiply numbers, you can also add and multiply vectors and matrices. And from there you can do complex operations between them. Linear Algebra is usually an undergraduate level course at a university.
- Calculus: At the most basic level, calculus involves studying how things change over time. Typically we start with motion and talk about speed, which is how an object changes its position over time. But speed itself can change over time, which refers to acceleration. This is the foundation of calculus. Typically, there are at least three levels of undergraduate courses.
- Probability and Statistics: AI merges the above mathematics with concepts found today in big data analysis. Learning data analysis starts with learning probability and statistics. There are usually at least two undergrad courses here.
As you learn these mathematics concepts, you’ll want to simultaneously learn how to do them numerically using different C++ libraries. One important such library is called Eigen.
While mastering these math topics, you can study the actual algorithms for AI, starting with:
- Machine learning algorithms: This goes beyond the concepts mentioned earlier and into how the algorithms actually work. There are many of these, including supervised learning, unsupervised learning, reinforcement learning, and deep learning. All of these are used today and play a part of training AI models. (For ideas on deep learning, you can find our rundown here.)
- Inference mechanics: This involves AI models being able to generate, or “infer” new information based on what the model already knows.
- Transformer architecture: This is the big one for today’s AI. It lays the groundwork for such AI systems as generative AI. (Indeed, the GPT in ChatGPT stands for Generative Pre-trained Transformer.)
And finally, you’ll definitely need to learn how to develop CUDA applications. As mentioned earlier, CUDA is the framework through which you can execute code directly on the GPUs. Remember, GPUs have potentially thousands of cores. But even targeting code that runs on just two cores requires a deep knowledge of what’s called parallel programming. Parallel programming techniques help you take a single set of code and distribute it among multiple cores, letting the code parts run simultaneously in a coordinated environment.
As such, you’ll need to learn both of those skills, parallel programming and CUDA programming. For parallel programming, you’ll need to study OpenCL. For CUDA, you can find information at NVIDIA’s site.
Pro tip: You’ll likely want to also learn how to provision EC2 servers on Amazon Web Services, as they have instances that have the most advanced NVIDIAGPUs attached to them. This is an ideal environment in which to practice CUDA programming and AI in general.
Conclusion
C++ is one of the more difficult languages to master for many software programmers, but because it’s so powerful, most of the AI libraries are written in it. That means if you master both AI concepts and the C++ language, a huge world will open up for you (and likely lots of job opportunities).
