When it comes to the infrastructure behind the AI craze, there are two things that are becoming paramount to the success of an AI Inference model working. Data and Latency. Are we going to process data at the edge or in the cloud? What about inference?
At AI Infrastructure Field Day 3, AIIFD3, a new company to me, called CTERA, talked about how they are solving those very issues, specifically when it comes to Latency. NVidia is solving the Data access issue with their server farms and Anthropic with MCP. CTERA is focused on the solving the latency issues.
Ever since I started this blog, Latency was the new Speedtest. 5G was supposed to push better latency compared 4G LTE. New versions of Wi-Fi are promising improved latency. When it comes to AI Infrastructure, it is the same, how do we reduce latency?
When it comes to Data, more and more of that data is unstructured. How do you handle that unstructured data at scale at speeds needed for the modern AI models? That is what CTERA is hoping to solve.
The CTERA Intelligent Data Platform is built to improve performance and capacity of AI models using Object Storage in S3 buckets, but processing it closer to the edge.
Another buzzword in the industry is Namespaces. My Wi-Fi friends may have heard of Namespaces being used to be handled on the WLANPi with Orb. Orb is also excellent for measuring latency on a network from the view of a client.
CTERA is using a Global Namespace for managing the data across a campus, branches, and the edge to help reduce the overhead of all this unstructured data.
By building this global namespace, CTERA can then better protect the data being collected from ransomware and other attacks. It is handling all the data at the metadata level and relying on the cloud storage vendors such as Object Storage in S3 buckets.
CTERA is only passing around the metadata instead of all the data which reduces the overhead of transferring vast amounts of unstructured data between data centers. That is how they are achieving lower latency.
That is an interesting area to be watching when it comes to moving data over Cellular and Wi-Fi networks. This is partially why 5G has not been as beneficial as promised. The use cases have found other mechanisms to handle the transfer of data and are not relying on the wild west of network connectivity solutions at the edge over cellular or satellite or Wi-Fi.
I recommend anyone interested in learning about how CTERA is reducing the latency for AI, watch their presentation at Tech Field Day’s AI Infrastructure Field Day.