AUTOMATIC LOW LEVEL OPERATOR LOOP GENERATON, PARALLELIZATION AND VECTORIZATION FOR TENSOR COMPUTATIONS

A method is provided for transforming a high-level language representation of a tensor computation graph into a low level language. The method includes assigning a tensor shape and a loop primitive. The method also includes generating, from the tensor computation graph and the assigned loop primitives, an initial loop structure. The method further includes positioning the layers of the tensor computation graph within a nested loop structure to provide a final loop structure, collapsing loops in the final loop structure, and mapping the collapsed loops to hardware components configured to execute the collapsed loops. The method can be applied to artificial intelligence (AI) and machine learning (ML) use cases for improved optimization of neural networks including compilation optimization for improving performance of simulations such as medical simulations, healthcare simulations, weather simulations, and/or simulations related to other complex systems, which can also support decision making.

DYNAMIC PATHWAYS FOR ARTIFICIAL INTELLIGENCE AND TENSOR COMPUTATION GRAPHS

A method for optimizing control flow in compiled computation graphs includes defining an intermediate representation (IR) of a computation graph, the computation graph IR including a main computation graph having at least one control flow primitive layer node pointing to one or more control flow sub-graph nodes. Fusable layer nodes of the main computation graph are identified and removed from the main computation graph, and the removed fusable layer nodes are duplicated into each of the one or more control flow sub-graph nodes. The method can be applied to machine learning frameworks, for example, for scientific computations such as in medical AI.

PREPROCESSING CODE USING LARGE LANGUAGE MODELS FOR PERFORMANCE PORTABILITY

A computer-implemented, machine learning method for preprocessing code for performance portability includes extracting performance critical code segments from an application and obtaining input data. Ground truth data is generated based on the input data and the application. Original code of the application is transpiled using a large language model (LLM) into a tensor computation language (TCL) candidate. Correctness of an implementation of the TCL candidate is verified using the ground truth data. The method has applications including, but not limited to, use cases in medical/healthcare, and other artificial intelligence applications for preprocessing and optimizing code for performance portability. The method can also support decision making and could be implemented with machine learning.

Facilitate high-performance hardware integration into AI Frameworks with the NEC SOL AI compiler

AI development has become increasingly driven by powerful frameworks like PyTorch and TensorFlow, supported by major tech companies. However, the rapid release cycles of these frameworks – every 3-6 months – pose a challenge for new hardware vendors. They struggle to develop the necessary AI functionality and keep pace with frequent updates. In this talk, we introduce NEC’s SOL AI compiler, which seamlessly integrates with PyTorch, TensorFlow, ONNX, Numpy, and soon JAX. SOL provides a unified compiler engine for these frameworks, supporting both inference and training, while also enabling model export to standalone libraries with minimal dependencies. Designed for device-agnostic support and ease of maintenance, SOL requires no specific compiler support (e.g., OpenCL, SyCL, OpenMP, Triton, MLIR, …) but can generate device tailored code with minimal coding effort. We will present SOL’s key concepts and its device-agnostic design in this talk.

COMBINATION OF MULTIPLE DATA PROCSSING AND MACHINE LEARNING FRAMEWORKS FOR A TARGET HARDWARE

A method for combining multiple different data processing, artificial intelligence and/or machine learning frameworks for execution by a target hardware includes extracting one or more computation graphs from each of the different frameworks. The computation graphs are combined into a fused computation graph. Memcopy operations are removed at edges between the computation graphs of the different frameworks. Memory spaces for computations in the fused computation graph are remapped to memory spaces of the target hardware.

ACCELERATION OF NEURAL NETWORKS USING DEPTH - FIRST PROCESSING

A method for accelerating a neural network includes identifying neural network layers that meet a locality constraint. Code is generated to implement depth-first processing for different hardware based on the identified neural network layers. The generated code is used to perform the depth-first processing on the neural network based on the generated code.

SOL: Reducing the Maintenance Overhead for Integrating Hardware Support into AI Frameworks

The increased interest in Artificial Intelligence (AI) raised the need for highly optimized and sophisticated AI frameworks. Starting with the Lua-based Torch many frameworks have emerged over time, such as Theano, Caffe, Chainer, CNTK, MxNet, PyTorch, DL4J, or TensorFlow.

All of these provide a high level scripting API that allows users to easily design neural networks and run these on various kinds of hardware. What the user usually does not see is the high effort put into these frameworks to provide peak execution performance.

Continue reading at nec.com

SOL: Single middleware for optimized multi-architecture AI training and deployment

NEC User Group Meeting