If you ask most people in IT which programming language is most commonly used for AI applications, they’ll almost all answer Python. Which raises the obvious questions: why, and why not Java instead? 

Python is actually older than Java by over 4 years, so its AI popularity is not because it is newer than other alternatives. Python’s origins have nothing to do with AI (or even numerically intensive applications). It was originally developed as a high-level scripting language for writing system utilities and applications on the Amoeba distributed operating system (which I doubt most people have even heard of). Python is also a poor choice for numerically intensive operations, which is a lot of what AI and LLMs need to do.

The real reason for Python’s popularity is that the mathematicians working on AI found it easy to learn. It could serve as a simple interface to high-performance computing libraries, typically written in complex low-level languages such as C and C++. This initial popularity led to many industry-standard frameworks, such as PyTorch and TensorFlow, being written in Python.

Does this mean Java has no place in AI?

Far from it, as was demonstrated recently by Azul’s AI4J online conference. This brought together nine AI proponents, most of whom are Java Champions, to present various ways Java can be used in this area.

The general theme that came across was that “Java owns the enterprise data”. Given Java’s enduring popularity for server-side applications, this is a statement that’s hard to dispute. This underscores the importance of Java when considering Retrieval Augmented Generation (RAG). Training an LLM uses a wide array of data and can deliver a model that is easy to communicate with in natural language. To change this from being smart to helpful (another quote from the conference), you need to access enterprise data in real time. The most valuable data for an organisation is contained in systems like those for ERP and CRM, as well as a plethora of databases. Most of that data is managed by Java-based applications.

What we end up with is the usual layered IT architecture, each layer providing an abstraction of the one below it. As was explained in one session, the LLM sits at the bottom, with vector search (or naïve RAG) above that and Graph RAG above that. 

Java delivers powerful technologies

Java can deliver powerful technologies that fit well into this critical space. Instead of searching flat text files, a RAG system can use tools like Neo4j and Cyrock.AI to model complex relationships among concepts, people, and documents. This allows the AI to fetch deep, interconnected context that standard vector databases might miss. Enterprise Java application platforms like Jakarta EE and Spring are rapidly adapting to the needs of AI systems; Josh Long of Pivotal and James Ward of Amazon provided details of how Spring and AWS Bedrock can be used in this way.

Another significant takeaway from the AI4J sessions was the use of predictive, or analytical AI rather than the high-profile generative AI (or GenAI). One session quoted a report from Deloitte on the State of AI that said that “30% of GenAI projects will be abandoned due to a lack of clear business value”. Predictive AI has many benefits for an enterprise, not least of which is its deterministic quality. This means it does not hallucinate (which you really don’t want when making mission-critical, enterprise-wide decisions).

AI with context

Context for AI was another recurring theme in these sessions. Better coordination of input to an LLM-based AI system is critical to success. Providing that bridge from smart to helpful is what enterprise users need the most.

This, in turn, led to the conclusion that one of the most important aspects of any AI system is performance. In its over thirty-year history, Java has gone from a platform teased for being slow to one that, using techniques like just-in-time (JIT) compilation, can even outperform native C and C++ code.

The AI4J sessions discussed a number of ways that Java delivers optimum performance:

• Project Panama for easier Java integration with lower-level C++ frameworks such as CUDA and ONNX Runtime.
• Virtual threads, capable of delivering massively greater scalability for applications that spend a lot of time blocking, which is what AI applications do as they wait for tokens to be processed.
• Project Babylon, which is exploring ways to target specific hardware architectures at runtime. Moving beyond JIT, this allows code to be targeted at GPUs or even FPGAs as and when necessary.
• Azul’s Prime JVM that delivers lower latency and higher throughput through different garbage collection (GC) and JIT implementations. All whilst maintaining full adherence to the Java SE specification, making it a drop-in replacement for other JVMs. No code changes or recompilations required.

Overall, the AI4J conference provided a fascinating insight into how the Java ecosystem is adapting (quickly) to the needs of AI applications.

About Simon Ritter

Simon has been in the IT business since 1984 and holds a Bachelor of Science degree in Physics from Brunel University in the UK. Simon joined Sun Microsystems in 1996 and started working with Java technology from JDK 1.0; he has spent time working in both Java development and consultancy. Having moved to Oracle as part of the Sun acquisition, he managed the Java Evangelism team for the core Java platform. Now at Azul, he continues to help people understand Java as well as Azul’s JVM technologies and products. Simon has twice been awarded Java Rockstar status at JavaOne and is a Java Champion. He represents Azul on the Java SE Expert Group, OpenJDK Vulnerability Group and Adoptium Steering Committee. He is also the author of OpenJDK Migration for Dummies.

About Azul

Azul is the trusted leader in enterprise Java for today’s AI and cloud-first world. Its open source-based Java platform empowers organizations to optimize the entire Java lifecycle to accelerate performance, strengthen security, reduce licensing and cloud costs, and boost developer productivity. Azul powers mission-critical systems for 36% of the Fortune 100, 50% of the Forbes Top Ten World’s Most Valuable Brands, and the world’s top 10 financial trading companies. Learn more at azul.com and follow @azulsystems.​