Tech
What are the storage requirements for AI training and inference? | Computer Weekly
Despite ongoing speculation around an investment bubble that may be set to burst, artificial intelligence (AI) technology is here to stay. And while an over-inflated market may exist at the level of the suppliers, AI is well-developed and has a firm foothold among organisations of all sizes.
But AI workloads place specific demands on IT infrastructure and on storage in particular. Data volumes can start big and then balloon, in particular during training phases as data is vectorised and checkpoints are created. Meanwhile, data must be curated, gathered and managed throughout its lifecycle.
In this article, we look at the key characteristics of AI workloads, the particular demands of training and inference on storage I/O, throughput and capacity, whether to choose object or file storage, and the storage requirements of agentic AI.
What are the key characteristics of AI workloads?
AI workloads can be broadly categorised into two key stages – training and inference.
During training, processing focuses on what is effectively pattern recognition. Large volumes of data are examined by an algorithm – likely part of a deep learning framework like TensorFlow or PyTorch – that aims to recognise features within the data.
This could be visual elements in an image or particular words or patterns of words within documents. These features, which might fall under the broad categories of “a cat” or “litigation”, for example, are given values and stored in a vector database.
The assigned values provide for further detail. So, for example “a tortoiseshell cat”, would comprise discrete values for “cat” and “tortoiseshell”, that make up the whole concept and allow comparison and calculation between images.
Once the AI system is trained on its data, it can then be used for inference – literally, to infer a result from production data that can be put to use for the organisation.
So, for example, we may have an animal tracking camera and we want it to alert us when a tortoiseshell cat crosses our garden. To do that it would infer the presence or not of a cat and whether it is tortoiseshell by reference to the dataset built during the training described above.
But, while AI processing falls into these two broad categories, it is not necessarily so clear cut in real life. It will always be the case that training will be done on an initial dataset. But after that it is likely that while inference is an ongoing process, training also becomes perpetual as new data is ingested and new inference results from it.
So, to labour the example, our cat-garden-camera system may record new cats of unknown types and begin to categorise their features and add them to the model.
What are the key impacts on data storage of AI processing?
At the heart of AI hardware are specialised chips called graphics processing units (GPUs). These do the grunt processing work of training and are incredibly powerful, costly and often difficult to procure. For these reasons their utilisation rates are a major operational IT consideration – storage must be able to handle their I/O demands so they are optimally used.
Therefore, data storage that feeds GPUs during training must be fast, so it’s almost certainly going to be built with flash storage arrays.
Another key consideration is capacity. That’s because AI datasets can start big and get much bigger. As datasets undergo training, the conversion of raw information into vector data can see data volumes expand by up to 10 times.
Also, during training, checkpointing is carried out at regular intervals, often after every “epoch” or pass through the training data, or after changes are made to parameters.
Checkpoints are similar to snapshots, and allow training to be rolled back to a point in time if something goes wrong so that existing processing does not go to waste. Checkpointing can add significant data volume to storage requirements.
So, sufficient storage capacity must be available, and will often need to scale rapidly.
What are the key impacts of AI processing on I/O and capacity in data storage?
The I/O demands of AI processing on storage are huge. It is often the case that model data in use will just not fit into a single GPU memory and so is parallelised across many of them.
Also, AI workloads and I/O differ significantly between training and inference. As we’ve seen, the massive parallel processing involved in training requires low latency and high throughput.
While low latency is a universal requirement during training, throughput demands may differ depending on the deep learning framework used. PyTorch, for example, stores model data as a large number of small files while TensorFlow uses a smaller number of large model files.
The model used can also impact capacity requirements. TensorFlow checkpointing tends towards larger file sizes, plus dependent data states and metadata, while PyTorch checkpointing can be more lightweight. TensorFlow deployments tend to have a larger storage footprint generally.
If the model is parallelised across numerous GPUs this has an effect on checkpoint writes and restores that mean storage I/O must be up to the job.
Does AI processing prefer file or object storage?
While AI infrastructure isn’t necessarily tied to one or other storage access method, object storage has a lot going for it.
Most enterprise data is unstructured data and exists at scale, and it is often what AI has to work with. Object storage is supremely well suited to unstructured data because of its ability to scale. It also comes with rich metadata capabilities that can help data discovery and classification before AI processing begins in earnest.
File storage stores data in a tree-like hierarchy of files and folders. That can become unwieldy to access at scale. Object storage, by contrast, stores data in a “flat” structure, by unique identifier, with rich metadata. It can mimic file and folder-like structures by addition of metadata labels, which many will be familiar with in cloud-based systems such as Google Drive, Microsoft OneDrive and so on.
Object storage can, however, be slow to access and lacks file-locking capability, though this is likely to be of less concern for AI workloads.
What impact will agentic AI have on storage infrastructure?
Agentic AI uses autonomous AI agents that can carry out specific tasks without human oversight. They are tasked with autonomous decision-making within specific, predetermined boundaries.
Examples would include the use of agents in IT security to scan for threats and take action without human involvement, to spot and initiate actions in a supply chain, or in a call centre to analyse customer sentiment, review order history and respond to customer needs.
Agentic AI is largely an inference phase phenomenon so compute infrastructure will not need to be up to training-type workloads. Having said that, agentic AI agents will potentially access multiple data sources across on-premises systems and the cloud. That will cover the range of potential types of storage in terms of performance.
But, to work at its best, agentic AI will need high-performance, enterprise-class storage that can handle a wide variety of data types with low latency and with the ability to scale rapidly. That’s not to say datasets in less performant storage cannot form part of the agentic infrastructure. But if you want your agents to work at their best you’ll need to provide the best storage you can.
Peter Thiel—the billionaire venture capitalist, PayPal and Palantir cofounder, and outspoken commentator on all matters relating to the “Antichrist”—appears at least 2,200 times in the latest batch of files released by the Department of Justice related to convicted sex offender and disgraced financier Jeffrey Epstein.
The tranche of records demonstrate how Epstein managed to cultivate an extensive network of wealthy and influential figures in Silicon Valley. A number of them, including Thiel, continued to interact with Epstein even after his 2008 guilty plea for solicitation of prostitution and of procurement of minors to engage in prostitution.
The new files show that Thiel arranged to meet with Epstein several times between 2014 and 2017. “What are you up to on Friday?” Thiel wrote to Epstein on April 5, 2016. “Should we try for lunch?” The bulk of the communications between the two men in the data dump concern scheduling meals, calls, and meetings with one another. Thiel did not immediately return a request for comment from WIRED.
One piece of correspondence stands out for being particularly bizarre. On February 3, 2016, Thiel’s former chief of staff and senior executive assistant, Alisa Bekins, sent an email with the subject line “Meeting – Feb 4 – 9:30 AM – Peter Thiel dietary restrictions – CONFIDENTIAL.” The initial recipient of the email is redacted, but it was later forwarded directly to Epstein.
The contents of the message are also redacted in at least one version of the email chain uploaded by the Justice Department on Friday. However, two other files from what appears to be the same set of messages have less information redacted.
In one email, Bekins listed some two dozen approved kinds of sushi and animal protein, 14 approved vegetables, and 0 approved fruits for Thiel to eat. “Fresh herbs” and “olive oil” were permitted, however, ketchup, mayonnaise, and soy sauce should be avoided. Only one actual meal was explicitly outlined: “egg whites or greens/salad with some form of protein,” such as steak, which Bekins included “in the event they eat breakfast.” It’s unclear if the February 4 meeting ultimately occurred; other emails indicate Thiel got stuck in traffic on his way to meet Epstein that day.
According to a recording of an undated conversation between Epstein and former Israeli Prime Minister Ehud Barak that was also part of the files the DOJ released on Friday, Epstein told Barak that he was hoping to meet Thiel the following week. He added that he was familiar with Thiel’s company Palantir, but proceeded to spell it out loud for Barak as “Pallentier.” Epstein speculated that Thiel may put Barak on the board of Palantir, though there’s no evidence that ever occurred.
“I’ve never met Peter Thiel, and everybody says he sort of jumps around and acts really strange, like he’s on drugs,” Epstein said at one point in the audio recording, referring to Thiel. The former prime minister expressed agreement with Epstein’s assessment.
In 2015 and 2016, Epstein put $40 million in two funds managed by one of Thiel’s investment firms, Valar Ventures, according to The New York Times. Epstein and Thiel continued to communicate and were discussing meeting with one another as recently as January 2019, according to the files released by the DOJ. Epstein committed suicide in his prison cell in August of that year.
Below are Thiel’s dietary restrictions as outlined in the February 2016 email. (The following list has been reformatted slightly for clarity.)
Elon Musk’s rocket and satellite company SpaceX is acquiring his AI startup xAI, the centibillionaire announced on Monday. In a blog post, Musk said the acquisition was warranted because global electricity demand for AI cannot be met with “terrestrial solutions,” and Silicon Valley will soon need to build data centers in space to power its AI ambitions.
“In the long term, space-based AI is obviously the only way to scale,” Musk wrote. “The only logical solution therefore is to transport these resource-intensive efforts to a location with vast power and space. I mean, space is called ‘space’ for a reason.”
The deal, which pulls together two of Musk’s largest private ventures, values the combined entity at $1.25 trillion, making it the most valuable private company in the world, according to a report from Bloomberg.
SpaceX was in the process of preparing to go public later this year before the xAI acquisition was announced. The space firm’s plans for an initial public offering are still on, according to Bloomberg.
In December, SpaceX told employees that it would buy insider shares in a deal that would value the rocket company at $800 billion, according to The New York Times. Last month, xAI announced that it had raised $20 billion from investors, bringing the company’s valuation to roughly $230 billion.
This isn’t the first time Musk has sought to consolidate parts of his vast business empire, which is largely privately owned and includes xAI, SpaceX, the brain interface company Neuralink, and the tunnel transportation firm the Boring Company.
Last year, xAI acquired Musk’s social media platform, X, formerly known as Twitter, in a deal that valued the combined entity at more than $110 billion. Since then, xAI’s core product, Grok, has become further integrated into the social media platform. Grok is featured prominently in various X features, and Musk has claimed the app’s content-recommendation algorithm is powered by xAI’s technology.
A decade ago, Musk also used shares of his electric car company Tesla to purchase SolarCity, a renewable energy firm that was run at the time by cousin Lyndon Rive.
The xAI acquisition demonstrates how Musk can use his expansive network of companies to help power his own often grandiose visions of the future. Elon Musk said in the blog post that SpaceX will immediately focus on launching satellites into space to power AI development on Earth, but eventually, the space-based data centers he envisions building could power civilizations on other planets, such as Mars.
“This marks not just the next chapter, but the next book in SpaceX and xAI’s mission: scaling to make a sentient sun to understand the Universe and extend the light of consciousness to the stars,” Musk said in the blog post.
Since last March, the Department of Health and Human Services has been using AI tools from Palantir to screen and audit grants, grant applications, and job descriptions for noncompliance with President Donald Trump’s executive orders targeting “gender ideology” and anything related to diversity, equity, and inclusion (DEI), according to a recently published inventory of all use cases HHS had for AI in 2025.
Neither Palantir nor HHS has publicly announced that the company’s software was being used for these purposes. During the first year of Trump’s second term, Palantir earned more than $35 million in payments and obligations from HHS alone. None of the descriptions for these transactions mention this work targeting DEI or “gender ideology.”
The audits have been taking place within HHS’s Administration for Children and Families (ACF), which funds family and child welfare and oversees the foster and adoption systems. Palantir is the sole contractor charged with making a list of “position descriptions that may need to be adjusted for alignment with recent executive orders.”
In addition to Palantir, the startup Credal AI—which was founded by two Palantir alumni—helped ACF audit “existing grants and new grant applications.” The “AI-based” grant review process, the inventory says, “reviews application submission files and generates initial flags and priorities for discussion.” All relevant information is then routed to the ACF Program Office for final review.
ACF staffers ultimately review any job descriptions, grants, and grant applications that are flagged by AI during a “final review” stage, according to the inventory. It also says that these particular AI use cases are currently “deployed” within ACF, meaning that they are actively being used at the agency.
Last year, ACF paid Credal AI about $750,000 to provide the company’s “Tech Enterprise Generative Artificial Intelligence (GenAI) Platform,” but the payment descriptions in the Federal Register do not mention DEI or “gender ideology.”
HHS, ACF, Palantir, and Credal AI did not return WIRED’s requests for comment.
The executive orders—Executive Order 14151, “Ending Radical and Wasteful Government DEI Programs and Preferencing,” and Executive Order 14168, “Defending Women From Gender Ideology Extremism and Restoring Biological Truth to the Federal Government”—were both issued on Trump’s first day in office last year.
The first of these orders demands an end to any policies, programs, contracts, grants that mention or concern DEIA, DEI, “equity,” or “environmental justice,” and charges the Office of Management and Budget, the Office of Personnel Management, and the attorney general with leading these efforts.
The second order demands that all “interpretation of and application” of federal laws and policies define “sex” as an “immutable biological classification” and define the only genders as “male” and “female.” It deems “gender ideology” and “gender identity” to be “false” and “disconnected from biological reality.” It also says that no federal funds can be used “to promote gender ideology.”
“Each agency shall assess grant conditions and grantee preferences and ensure grant funds do not promote gender ideology,” it reads.
The consequences of Executive Order 14151, targeting DEI, and Executive Order 14168, targeting “gender ideology,” have been felt deeply throughout the country over the past year.
Early last year, the National Science Foundation started to flag any research that contained terms associated with DEI—including relatively general terms, like “female,” “inclusion,” “systemic,” or “underrepresented”—and place it under official review. The Centers for Disease Control and Prevention began retracting or pausing research that mentioned terms like “LGBT,” “transsexual,” or “nonbinary,” and stopped processing any data related to transgender people. Last July, the Substance Abuse and Mental Health Services Administration removed an LGBTQ youth service line offered by the 988 Suicide & Crisis Lifeline.
Why Are Gold Prices Swinging? Nirmala Sitharaman Breaks It Down
Intertextile Shanghai 2026 to debut pet boutique zone
Patriots’ Drake Maye ranks wife’s viral TikTok baking recipes ahead of Super Bowl LX
-
Sports7 days agoPSL 11: Local players’ category renewals unveiled ahead of auction
-
Entertainment6 days agoClaire Danes reveals how she reacted to pregnancy at 44
-
Business7 days agoBanking services disrupted as bank employees go on nationwide strike demanding five-day work week
-
Sports6 days agoCollege football’s top 100 games of the 2025 season
-
Fashion1 week agoSpain’s apparel imports up 7.10% in Jan-Oct as sourcing realigns
-
Sports6 days agoTammy Abraham joins Aston Villa 1 day after Besiktas transfer
-
Politics6 days agoTrump vows to ‘de-escalate’ after Minneapolis shootings
-
Entertainment6 days agoK-Pop star Rosé to appear in special podcast before Grammy’s