Tech
OpenAI Is Asking Contractors to Upload Work From Past Jobs to Evaluate the Performance of AI Agents
OpenAI is asking third-party contractors to upload real assignments and tasks from their current or previous workplaces so that it can use the data to evaluate the performance of its next-generation AI models, according to records from OpenAI and the training data company Handshake AI obtained by WIRED.
The project appears to be part of OpenAI’s efforts to establish a human baseline for different tasks that can then be compared with AI models. In September, the company launched a new evaluation process to measure the performance of its AI models against human professionals across a variety of industries. OpenAI says this is a key indicator of its progress towards achieving AGI, or an AI system that outperforms humans at most economically valuable tasks.
“We’ve hired folks across occupations to help collect real-world tasks modeled off those you’ve done in your full-time jobs, so we can measure how well AI models perform on those tasks,” reads one confidential document from OpenAI. “Take existing pieces of long-term or complex work (hours or days+) that you’ve done in your occupation and turn each into a task.”
OpenAI is asking contractors to describe tasks they’ve done in their current job or in the past and to upload real examples of work they did, according to an OpenAI presentation about the project viewed by WIRED. Each of the examples should be “a concrete output (not a summary of the file, but the actual file), e.g., Word doc, PDF, Powerpoint, Excel, image, repo,” the presentation notes. OpenAI says people can also share fabricated work examples created to demonstrate how they would realistically respond in specific scenarios.
OpenAI and Handshake AI declined to comment.
Real-world tasks have two components, according to the OpenAI presentation. There’s the task request (what a person’s manager or colleague told them to do) and the task deliverable (the actual work they produced in response to that request). The company emphasizes multiple times in instructions that the examples contractors share should reflect “real, on-the-job work” that the person has “actually done.”
One example in the OpenAI presentation outlines a task from a “Senior Lifestyle Manager at a luxury concierge company for ultra-high-net-worth individuals.” The goal is to “Prepare a short, 2-page PDF draft of a 7-day yacht trip overview to the Bahamas for a family who will be traveling there for the first time.” It includes additional details regarding the family’s interests and what the itinerary should look like. The “experienced human deliverable” then shows what the contractor in this case would upload: a real Bahamas itinerary created for a client.
OpenAI instructs the contractors to delete corporate intellectual property and personally identifiable information from the work files they upload. Under a section labeled “Important reminders,” OpenAI tells the workers to “Remove or anonymize any: personal information, proprietary or confidential data, material nonpublic information (e.g., internal strategy, unreleased product details).”
One of the files viewed by WIRED document mentions an ChatGPT tool called “Superstar Scrubbing” that provides advice on how to delete confidential information.
Evan Brown, an intellectual property lawyer with Neal & McDevitt, tells WIRED that AI labs that receive confidential information from contractors at this scale could be subject to trade secret misappropriation claims. Contractors who offer documents from their previous workplaces to an AI company, even scrubbed, could be at risk of violating their previous employers’ non-disclosure agreements, or exposing trade secrets.
“The AI lab is putting a lot of trust in its contractors to decide what is and isn’t confidential,” says Brown. “If they do let something slip through, are the AI labs really taking the time to determine what is and isn’t a trade secret? It seems to me that the AI lab is putting itself at great risk.”
I tried the battery version, which does require you recharge it every couple of weeks, but the wired-in version is the top recommendation on our guide to the Best Video Doorbells.
A Better Birdhouse
I had a new-to-me problem this spring: bird invasion. A little bird made a nest in my front-door wreath without us noticing. One evening, my sister opened the door, and the bird flew out of the nest and straight into our house. After a 30-minute battle to get it outside again (and keep my cat from eating it), it wasn’t until we saw the bird fly off the door again the next day that we realized it was calling our home its home, too.
If this is a common problem at your house, our resident bird-gear tester Kat Merck has a solution: a smart nesting box. Birdfy makes a few different smart bird feeders we like for bird-watching, and the Nest Duo is a birdhouse that lets you watch the birds while they nest inside of it. It’s a slim, attractive box that will add to your front yard’s style while also packing two solar-powered cameras (one facing the entrance, one focused inside) so you can bird-watch from multiple angles. It comes with different hole sizes to appeal to different species, metal predator guards to prevent chewing around the hole, and a remote control to reset or recharge the camera without disturbing your feathered neighbors.
Stylish Smart Lights
Govee
Outdoor Clear Bulb String Lights
I’ve liked Govee’s smart outdoor string lights before, usually for my holiday decor, and have previously recommended something similar with a bistro-light-like look that happened to be smart. These clear bulb string lights are part of Govee’s current lineup and have a contemporary twist with a triangle in the center instead of the wire filament. These are a fun option for outdoor lights you can enjoy on warm nights, and they can do every color and shade of white without looking as bulky as permanent outdoor lights. (Added bonus, these lights are also Matter compatible!)
Fresh Bulbs
Cync
Smart LED Light Bulb, PAR38
If you have light fixtures you want to remote-control, add an outdoor smart bulb. There are tons to choose from, and you can usually find one from any brand you already have at home. The only downside is that outdoor-rated smart bulbs are usually 4.75-inch-diameter PAR38-style bulbs, so they’re best for downward-facing floodlights on your porch or balcony. They’ll likely be too big to fit in a wall fixture as a replacement for a normal-sized bulb. Don’t just grab any smart bulb—not all are outdoor-rated. Check for mentions of outdoor use and waterproof ratings to make sure they’re safe to use. I’m a big fan of Cync bulbs, and the brand has an outdoor version of the Cync Full Color bulbs I like to use indoors. You’ll be able to add fun colors as well as shades of white, so you can turn the porch a spooky orange or red for Halloween, pink for Valentine’s Day, or the colors of your favorite sports team on game day.
Remote-Controlled Garage
If your garage is the centerpiece of your home’s curb appeal, you can control it as easily as a smart door by adding a smart controller. You can do two different styles: I have the Chamberlain MyQ professionally installed smart garage opener, which means the device that controls my garage has these smarts built into it (plus a camera, but I find it doesn’t work great with how far the device is from my Wi-Fi router), or you can get a smart garage controller that can add smart features onto an existing garage door. Both let you check whether the garage is open or closed and operate it remotely, and you can add a video keypad that doubles as a video doorbell and can let you open or close the garage without your phone.
Smart Shades
The front of my home faces west, so it’s absolutely baking at the end of the day. What I need to add are some of our favorite smart shades to automate closing the shades on that side of the house at the right time of day. These also give your home a nice, cohesive look and immediate, controllable privacy from the outside world. WIRED reviewer Simon Hill recommends the SmartWings shades as his top picks, and Lutron’s Caseta shades if you’re looking for a more upgraded look.
Invisible Swaps
Looking to add some smarts without touching your existing setup? These switch-ups can make your front door and yard smart without being visible.
Power up with unlimited access to WIRED. Get best-in-class reporting and exclusive subscriber content that’s too important to ignore. Subscribe Today.
April might be springtime in the northern hemisphere, but some of the best streaming services seem to think it’s the perfect time for a dry run of spooky season. How else to explain the arrival of some exquisitely dark slices of horror, like 28 Days Later: The Bone Temple arriving on Netflix, Weapons coming to Prime Video, or Shelby Oaks landing on Hulu? If you prefer your off-season Halloween viewing to be in the vein of campy B movies rather than serious scares though, horror specialist Shudder has you covered with Deathstalker, a gloriously cheesy reboot of a near-forgotten ’80s series.
Reality is often scarier than fiction though, as shown by Louis Theroux’s Inside the Manosphere—his first documentary film with Netflix, exploring the dark side of social media and the world of toxic male influencers. (Be sure to read our interview with the filmmaker.) And if the thought of that leaves you wanting something a bit more wholesome to watch, thankfully Zootopia 2 has popped up on Disney+—and there’s even a rabbit in that, for some appropriately springtime imagery.
Here are WIRED’s picks of the best movies to watch right now.
28 Years Later: The Bone Temple
The fourth film in the long-running postapocalyptic horror series switches focus from rampaging rage zombies to a more dangerous threat: humans. OK, OK, “people are the real monsters” isn’t a hot take for the genre, but The Bone Temple offers a unique twist, with 28 Years Later survivor Spike (Alfie Williams) trapped in the company of a murderous gang led by deranged satanist “Sir Lord” Jimmy Crystal (Sinners’ Jack O’Connell). The villain is modeled on disgraced British TV presenter Jimmy Savile, whose sexual abuse crimes hadn’t been revealed by the time of the initial outbreak in 28 Days Later, adding a dash of real-world terror.
As the group stalks what remains of the English countryside, Spike’s only hope might be Dr. Ian Kelson (Ralph Fiennes), whose experiments on curing alpha zombie Samson (Chi Lewis-Parry) might hold humanity’s last hope. Although best watched back to back with its predecessor for the full, horrifying picture, director Nia DaCosta’s chapter stands on its own—and earns bonus points for one of the best uses of Iron Maiden’s “Number of the Beast” in film history.
Louis Theroux: Inside the Manosphere
It’s the silence that does the trick; British documentarian Louis Theroux always knows when not to speak and instead let his subject expose themselves for the world to see. It’s a masterful technique whether Theroux is investigating the Westboro Baptist Church or UFO conspiracy theorists, but it is rarely put to better use than in his latest outing: exploring the online “manosphere” subculture of self-appointed “alphas” offering toxic advice on how to be a “real man.” Speaking with key figures in the loosely defined movement, Theroux’s mild-mannered approach often leaves them to do most of the talking, exposing shockingly misogynistic and extremist views. Even more distressing? The quiet revelation that for many of them their performative masculinity is all just one big grift, and how they rationalize the harm they cause in pursuit of a payout. Depressing but compelling viewing—not all men, but definitely all of these men.
Crime 101
Jewel thief Mike (Chris Hemsworth) is the best in the business, a meticulous planner who pulls off his heists without leaving a shred of evidence—much to the consternation of LAPD detective Lou Lubesnick (Mark Ruffalo), who doesn’t even know exactly who he’s hunting for a string of thefts. Elsewhere in the City of Angels, Sharon (Halle Berry) is an underappreciated VP at an insurance firm, frustrated at being passed over for promotion for years. She’s the perfect insider to help Mike orchestrate an elaborate $11 million diamond heist. But as Lou uncovers evidence connecting to Mike’s past, and the chaotic, violent biker Ormon (Barry Keoghan) aims to take the score for himself, even the most masterful planning can’t prevent everything spiraling dangerously out of control.
Kevin Weil, OpenAI’s former chief product officer who was recently tapped to build a new AI workspace for scientists, Prism, is leaving the company, WIRED has confirmed. Weil was previously an early executive leading product at Instagram.
OpenAI is also sunsetting Prism, which the company launched as a web app in January this year to give scientists a better way to work with AI. The company is folding the roughly 10-person team behind it into Thibault Sottiaux’s Codex team. An OpenAI spokesperson confirmed the changes, and tells WIRED this is part of the company’s effort to unify its business and product strategy. OpenAI has broader ambitions to turn Codex, its AI coding application, into an “everything app.”
Weil, who joined OpenAI in June 2024, announced last September that he would be starting a new initiative inside of the company called “OpenAI for Science.” Now, OpenAI is dispersing those employees throughout the company’s product, research, and infrastructure teams. An OpenAI spokesperson reiterated the company’s commitment to accelerating scientific discovery, and says it’s one of the clearest ways AI can benefit humanity.
OpenAI is currently trying to refocus the company around a few key areas, such as enterprise offerings and coding. Last month, OpenAI’s CEO of AGI deployment Fidji Simo told staff that the company needs to simplify its product offerings. The push to divert resources to more consequential efforts resulted in OpenAI discontinuing its Sora video-generation app.
This is a developing story. Please check back for updates.
Ariana Grande teases new era after ‘Eternal Sunshine’ with cryptic video
I don’t regret gifting Nobel prize to Trump: Venezuela’s Machado
NaBFID signs pact with PDCOR to expand advisory support for state projects – The Times of India
-
Politics1 week agoIndian airlines hit hardest after Dubai limits foreign flights until May 31
-
Entertainment6 days agoPalace left in shock as Prince William cancels grand ceremony
-
Sports5 days agoThe case for Man United’s Fernandes as Premier League’s best
-
Politics1 week agoChinese, Taiwanese will unite, Xi tells Taiwan opposition leader
-
Business5 days agoUK could adopt EU single market rules under new legislation
-
Entertainment1 week agoDua Lipa hits major career high ahead of wedding with Callum Turner
-
Business1 week agoThe FAA wants gamers to apply for air traffic control jobs
-
Sports1 week agoLamar Jackson hits back at critics with faithful message on social media