Connect with us

Tech

What are the storage requirements for AI training and inference? | Computer Weekly

Published

on

What are the storage requirements for AI training and inference? | Computer Weekly


Despite ongoing speculation around an investment bubble that may be set to burst, artificial intelligence (AI) technology is here to stay. And while an over-inflated market may exist at the level of the suppliers, AI is well-developed and has a firm foothold among organisations of all sizes.

But AI workloads place specific demands on IT infrastructure and on storage in particular. Data volumes can start big and then balloon, in particular during training phases as data is vectorised and checkpoints are created. Meanwhile, data must be curated, gathered and managed throughout its lifecycle.

In this article, we look at the key characteristics of AI workloads, the particular demands of training and inference on storage I/O, throughput and capacity, whether to choose object or file storage, and the storage requirements of agentic AI.

What are the key characteristics of AI workloads?

AI workloads can be broadly categorised into two key stages – training and inference.

During training, processing focuses on what is effectively pattern recognition. Large volumes of data are examined by an algorithm – likely part of a deep learning framework like TensorFlow or PyTorch – that aims to recognise features within the data.

This could be visual elements in an image or particular words or patterns of words within documents. These features, which might fall under the broad categories of “a cat” or “litigation”, for example, are given values and stored in a vector database.

The assigned values provide for further detail. So, for example “a tortoiseshell cat”, would comprise discrete values for “cat” and “tortoiseshell”, that make up the whole concept and allow comparison and calculation between images.

Once the AI system is trained on its data, it can then be used for inference – literally, to infer a result from production data that can be put to use for the organisation.

So, for example, we may have an animal tracking camera and we want it to alert us when a tortoiseshell cat crosses our garden. To do that it would infer the presence or not of a cat and whether it is tortoiseshell by reference to the dataset built during the training described above.

But, while AI processing falls into these two broad categories, it is not necessarily so clear cut in real life. It will always be the case that training will be done on an initial dataset. But after that it is likely that while inference is an ongoing process, training also becomes perpetual as new data is ingested and new inference results from it.

So, to labour the example, our cat-garden-camera system may record new cats of unknown types and begin to categorise their features and add them to the model.

What are the key impacts on data storage of AI processing?

At the heart of AI hardware are specialised chips called graphics processing units (GPUs). These do the grunt processing work of training and are incredibly powerful, costly and often difficult to procure. For these reasons their utilisation rates are a major operational IT consideration – storage must be able to handle their I/O demands so they are optimally used.

Therefore, data storage that feeds GPUs during training must be fast, so it’s almost certainly going to be built with flash storage arrays.

Another key consideration is capacity. That’s because AI datasets can start big and get much bigger. As datasets undergo training, the conversion of raw information into vector data can see data volumes expand by up to 10 times.

Also, during training, checkpointing is carried out at regular intervals, often after every “epoch” or pass through the training data, or after changes are made to parameters.

Checkpoints are similar to snapshots, and allow training to be rolled back to a point in time if something goes wrong so that existing processing does not go to waste. Checkpointing can add significant data volume to storage requirements.

So, sufficient storage capacity must be available, and will often need to scale rapidly.

What are the key impacts of AI processing on I/O and capacity in data storage?

The I/O demands of AI processing on storage are huge. It is often the case that model data in use will just not fit into a single GPU memory and so is parallelised across many of them.

Also, AI workloads and I/O differ significantly between training and inference. As we’ve seen, the massive parallel processing involved in training requires low latency and high throughput.

While low latency is a universal requirement during training, throughput demands may differ depending on the deep learning framework used. PyTorch, for example, stores model data as a large number of small files while TensorFlow uses a smaller number of large model files.

The model used can also impact capacity requirements. TensorFlow checkpointing tends towards larger file sizes, plus dependent data states and metadata, while PyTorch checkpointing can be more lightweight. TensorFlow deployments tend to have a larger storage footprint generally.

If the model is parallelised across numerous GPUs this has an effect on checkpoint writes and restores that mean storage I/O must be up to the job.

Does AI processing prefer file or object storage?

While AI infrastructure isn’t necessarily tied to one or other storage access method, object storage has a lot going for it.

Most enterprise data is unstructured data and exists at scale, and it is often what AI has to work with. Object storage is supremely well suited to unstructured data because of its ability to scale. It also comes with rich metadata capabilities that can help data discovery and classification before AI processing begins in earnest.

File storage stores data in a tree-like hierarchy of files and folders. That can become unwieldy to access at scale. Object storage, by contrast, stores data in a “flat” structure, by unique identifier, with rich metadata. It can mimic file and folder-like structures by addition of metadata labels, which many will be familiar with in cloud-based systems such as Google Drive, Microsoft OneDrive and so on.

Object storage can, however, be slow to access and lacks file-locking capability, though this is likely to be of less concern for AI workloads.

What impact will agentic AI have on storage infrastructure?

Agentic AI uses autonomous AI agents that can carry out specific tasks without human oversight. They are tasked with autonomous decision-making within specific, predetermined boundaries.

Examples would include the use of agents in IT security to scan for threats and take action without human involvement, to spot and initiate actions in a supply chain, or in a call centre to analyse customer sentiment, review order history and respond to customer needs.

Agentic AI is largely an inference phase phenomenon so compute infrastructure will not need to be up to training-type workloads. Having said that, agentic AI agents will potentially access multiple data sources across on-premises systems and the cloud. That will cover the range of potential types of storage in terms of performance.

But, to work at its best, agentic AI will need high-performance, enterprise-class storage that can handle a wide variety of data types with low latency and with the ability to scale rapidly. That’s not to say datasets in less performant storage cannot form part of the agentic infrastructure. But if you want your agents to work at their best you’ll need to provide the best storage you can.



Source link

Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Tech

Just in Time for Spring, Don’t Miss These Electric Scooter Deals

Published

on

Just in Time for Spring, Don’t Miss These Electric Scooter Deals


The snow is melting, the days are getting longer, and I can almost smell the springtime ahead. Soon, we’ll be cruising around town on ebikes and electric scooters instead of burning fossil fuels. For now, the weather hasn’t quite caught up, which is great for markdowns. Many of the best electric scooters are still seeing significant discounts. If you’ve been thinking about buying one, now’s the best time: prices are low, and sunny commuting days are just ahead.

Gear editor Julian Chokkattu has spent five years testing more than 45 electric scooters. These are his top picks that are also on sale right now.

Apollo Go for $849 ($450 Off)

Photograph: Julian Chokkattu

This is Gear editor Julian Chokkattu’s favorite scooter. The riding experience is powerful and smooth, thanks to its dual 350-watt motors and solid front and rear suspensions. The speed maxes out at 28 miles per hour (mph), which doesn’t make it the fastest scooter on the market, but it has a good range. (Chokkattu is a very tall man and was able to travel 15 miles on a single charge at 15 mph.) Other Apollo features he appreciates: turn signals, a dot display, a bell, along with a headlight and an LED strip for extra visibility.

Apollo Phantom 2.0 for $2099 ($900 Off)

  • Photograph: Julian Chokkattu

  • Photograph: Julian Chokkattu

  • Photograph: Julian Chokkattu

The Apollo Phantom 2.0 maxes out at 44 mph, with plenty of power from its dual 1,750-watt motors. It’s a gorgeous scooter, designed with 11-inch self-healing tubeless tires and a dual-spring suspension system for a smooth riding experience. But with great power comes great weight. At 102 pounds, the Phantom 2.0 is the heaviest electric scooter Chokkattu has tested, so I would only recommend this purchase if you don’t live in a walkup and/or have a garage.

More Discounted Electric Scooters

Segway

Max G3

This is the best commuter scooter, with more power and range than the Apollo Go and a fast 3.5-hour recharge time.

Segway

Ninebot F3 Electric Scooter

The Segway F3 is designed with turn signals, a bell, a bright display, and a feature-rich app experience.

Niu KQi 300X

This is the best all-terrain scooter, with reliable suspension, dual disc brakes, and thick 10.5-inch tubeless tires.

Segway

E2 Pro

This is the best budget scooter, designed with a decent 350-watt motor, a max speed of 15 mph, a front drum brake, and a rear electronic brake.



Source link

Continue Reading

Tech

What’s an E-Bike? California Wants You to Know

Published

on

What’s an E-Bike? California Wants You to Know


A few months ago, a family came into Pasadena Cyclery in Pasadena, California, for a repair on what they thought was their teenager’s e-bike. “I can’t fix that here,’ Daniel Purnell, a store manager and technician, remembers telling them. “That’s a motorcycle.” The mother got upset. She didn’t realize that what she thought was an e-bike could go much faster, perhaps up to 55 miles per hour.

“There’s definitely an education problem,” Purnell says. In California, bike advocates are pushing a new bill designed to clear up that confusion around what counts as an electric bicycle—and what doesn’t.

It’s a tricky balance. On one hand, backers want to allow riders access to new, faster, and more affordable non-car transportation options, ones that don’t require licenses and are emission-free. On the other hand, people, and especially kids, seem to be getting hurt. E-bike-related injuries jumped more than 1,020 percent nationwide between 2020 and 2024, according to hospital data, though it’s not clear if the stats-keepers can routinely distinguish between e-bikes and their faster, “e-moto” cousins. (Moped and powered-assisted cycle injuries jumped 67 percent in that same period.)

“We’re overdue to have better e-bike regulation,” says California state senator Catherine Blakespear, a Democrat who sponsored the bill and represents parts of North County in San Diego. “This has been an ongoing and growing issue for years.”

Senate Bill 1167 would make it illegal for retailers to label higher-powered, electric-powered vehicles as e-bikes. It would clarify that e-bikes have fully operative pedals and electric motors that don’t exceed 750 watts, enough to hit top speeds between 20 and 28 mph.

“We’re not against these devices,” says Kendra Ramsey, the executive director of the California Bicycle Coalition, which represents riders and is promoting the legislation. “People think they’re e-bikes and they’re not really e-bikes.”

Bill backers say they hope the fix, if it passes, makes a difference, especially for teenagers, who love the freedom that electric motors give them but can get into trouble if something goes wrong at higher speeds. Kids 17 and younger accounted for 20 percent of US e-bike injuries from 2020 to 2024, about in line with the share of the total population. But headlines—and the laws that follow them—have focused on teen injuries and even deaths.

There are no national laws governing e-bike riding. But bike backers spent years moving between states to pass laws that put e-bikes into three classes: Class 1, which have pedal-assist that only works when they’re actually pedaled, and goes up to 20 mph; Class 2, which have throttles that work without pedaling but still only reach 20 mph; and Class 3, which use pedal-assist to move up to 28 mph. Plenty of states and cities restrict the most powerful Class 3 bikes to people older than 16. (In a complicated twist, some e-bikes have different “modes,” allowing riders to toggle between Class 2 and Class 3.)

Last year, researchers visited 19 San Francisco Bay Area middle and high schools and found that 88 percent of the electric two-wheeled devices parked there were so high-powered and high-speed that they didn’t comply with the three-class system at all.

E-bikes have clearly struck a chord with state policymakers: At least 10 bills introduced this year deal with e-bikes, according to Ramsey.

Some bike advocates believe injuries have less to do with e-bikes than “e-motos,” a category that’s less likely to appear in retail stores or the sort of social media ads attracting teens to the tech. These have more powerful motors and can travel in excess of 30 mph. Vehicles, like the Surron Ultra Bee, which can hit top speeds of 55 mph, or Tuttio ICT, which can hit 50, are often marketed by retailers as “electric bikes.” Because so many sales happen online, it can be hard for people, and especially parents, to know what they’re getting into.



Source link

Continue Reading

Tech

OpenAI Fires an Employee for Prediction Market Insider Trading

Published

on

OpenAI Fires an Employee for Prediction Market Insider Trading


OpenAI has fired an employee following an investigation into their activity on prediction market platforms including Polymarket, WIRED has learned.

OpenAI CEO of Applications, Fidji Simo, disclosed the termination in an internal message to employees earlier this year. The employee, she said, “used confidential OpenAI information in connection with external prediction markets (e.g. Polymarket).”

“Our policies prohibit employees from using confidential OpenAI information for personal gain, including in prediction markets,” says spokesperson Kayla Wood. OpenAI has not revealed the name of the employee or the specifics of their trades.

Evidence suggests that this was not an isolated event. Polymarket runs on the Polygon blockchain network, so its trading ledger is pseudonymous but traceable. According to an analysis by the financial data platform Unusual Whales, there have been clusters of activities, which the service flagged as suspicious, around OpenAI-themed events since March 2023.

Unusual Whales flagged 77 positions in 60 wallet addresses as suspected insider trades, looking at the age of the account, trading history, and significance of investment, among other factors. Suspicious trades hinged on the release dates of products like Sora, GPT-5, and the ChatGPT Browser, as well as CEO Sam Altman’s employment status. In November 2023, two days after Altman was dramatically ousted from the company, a new wallet placed a significant bet that he would return, netting over $16,000 in profits. The account never placed another bet.

The behavior fits into patterns typical of insider trades. “The tell is the clustering. In the 40 hours before OpenAI launched its browser, 13 brand-new wallets with zero trading history appeared on the site for the first time to collectively bet $309,486 on the right outcome,” says Unusual Whales CEO Matt Saincome. “When you see that many fresh wallets making the same bet at the same time, it raises a real question about whether the secret is getting out.”

Prediction markets have exploded in popularity in recent years. These platforms allow customers to buy “event contracts” on the outcomes of future events ranging from the winner of the Super Bowl to the daily price of Bitcoin to whether the United States will go to war with Iran. There are a wide array of markets tied to events in the technology sector; you can trade on what Nvidia’s quarterly earnings will be, or when Tesla will launch a new car, or which AI companies will IPO in 2026.

As the platforms have grown, so have concerns that they allow traders to profit from insider knowledge. “This prediction market world makes the Wild West look tame in comparison,” says Jeff Edelstein, a senior analyst at the betting news site InGame. “If there’s a market that exists where the answer is known, somebody’s going to trade on it.”

Earlier this week, Kalshi announced that it had reported several suspicious insider trading cases to the Commodity Futures Trading Commission, the government agency overseeing these markets. In one instance, an employee of the popular YouTuber Mr. Beast was suspended for two years and fined $20,000 for making trades related to the streamer’s activities; in another, the far-right political candidate Kyle Langford was banned from the platform for making a trade on his own campaign. The company also announced a number of initiatives to prevent insider trading and market manipulation.

While Kalshi has heavily promoted its crackdown on insider trading, Polymarket has stayed silent on the matter. The company did not return requests for comments.

In the past, major trades on technology-themed markets have sparked speculation that there are Big Tech employees profiting by using their insider knowledge to gain an edge. One notorious example is the so-called “Google whale,” a pseudonymous account on Polymarket that made over $1 million trading on Google-related events, including a market on who the most-searched person of the year would be in 2025. (It was the singer D4vd, who is best known for his connection to an ongoing murder investigation after a young fan’s remains were found in a vehicle registered to him.)



Source link

Continue Reading

Trending