Show Notes
The people building the world’s most powerful artificial intelligence systems promise extraordinary gains. They also warn those systems could escape our control. That is the ethical fault line running through this week’s episode of The Business of Tech. My guest, New Zealander Peter Barnett, works at the Machine Intelligence Research Institute in Berkeley, California. The University of Otago physics graduate develops governance plans aimed at preventing catastrophic outcomes from advanced AI. He previously worked on technical AI alignment, the difficult job of making powerful systems act in line with human intentions. This conversation is a topical one. OpenAI is facing fresh turmoil over its approach to safety. Three recently fired safety and alignment researchers claim they were punished for challenging the company and working with outside evaluators. OpenAI rejects that account, saying they mishandled sensitive information and breached its trust. David Robinson, one of OpenAI’s senior safety figures, has also resigned. The former author of safety reports accompanying major model launches says the company’s culture is “broken” and the industry is not being nearly careful enough. He just sat down for a podcast interview with Ezra Klein of the New York Times, which makes for fascinating viewing. These departures expose the moral dilemma at the centre of the AI boom. Worrying emergent traits in AI The companies racing towards more capable models say the technology could cure disease, lift productivity and create enormous wealth. But they face fierce competition, huge investor expectations and a geopolitical contest framed around beating China. Every incentive rewards speed. Caution risks handing the advantage to a rival. So who decides when the danger is too great? Barnett argues it cannot be the companies themselves. He says frontier AI developers are effectively writing their own rules and can loosen them when safety commitments obstruct progress. Independent audits, government oversight and limits on training more powerful models may all be needed. His concern is not simply that people will misuse AI. It is that increasingly autonomous systems could learn to game evaluations, work around safeguards and pursue goals their creators did not intend. Safety researchers still cannot reliably predict everything a frontier model will be capable of before it is trained. Barnett believes New Zealand can play a role. We do not build frontier models, but our nuclear-free history shows a small country can challenge powerful nations on an issue carrying global risk. When insiders abandon lucrative careers to raise the alarm, maybe it's worth listening to them. Listen to the full conversation with Peter Barnett on The Business of Tech, streaming on iHeartRadio, Apple, Spotify or your podcast app of choice. See omnystudio.com/listener for privacy information.
More from The SME Stream
Inflation is still much higher than it should be, largely because of high global oil prices. Stats NZ data shows the rate's now on 3.7%, down from 4.1% three months ago. Domestic non-tradeable inflation —for things like housing, electricity, and healthcare— has been trending down. Internationally-driven tradeable inflation —for things we import and export— has dipped, after trending up. But NZ Herald Business Editor at Large Liam Dann told Kerre Woodham economists are forecasting anywhere up to 4.7% inflation for the December quarter, which is pretty ugly. He says even if the Reserve Bank were to look through the impact of oil prices, the Consumer Price Index would still rise around 3.2%, which they can't ignore. Dann says it's one of the reasons economists expect them to hike interest rates. LISTEN ABOVE See omnystudio.com/listener for privacy information.
We may be at a turning point when it comes to property investment. Cotality data shows residential property is worth $1.66 trillion and still accounts for more than 50% of all household assets. But it also shows property sales and values are largely subdued. Chief Property Economist Kelvin Davidson told Kerre Woodham over the last couple of decades, strong capital gains and low interest rates were a big driver for the housing market. He says mortgage costs went down and selling provided capital gains as well, but this potentially won't last into the future. Davidson says property investment will probably need to be done a little bit differently, focusing on active management and cash flow rather than relying on capital gains. LISTEN ABOVE See omnystudio.com/listener for privacy information.
Inflation's eased up slightly - but it's also become more persistent and widespread. The official inflation rate has fallen in the past three months - from 4.1 percent to 3.7 percent - due to a brief fall in fuel prices. But underlying core inflation has been rising and more items are increasing in price. Labour's Finance Spokesperson Barbara Edmonds told Francesca Rudkin the government doesn't have a plan to tackle the rising prices. LISTEN ABOVE See omnystudio.com/listener for privacy information.
National and Labour are offering quite different solutions to persistently high inflation. Headline inflation's eased back to 3.7%, but some economists say it could reach as high as 4.7% by the end of the year. Labour's Barbara Edmonds says freezing the fuel tax would address the single biggest factor – surging fuel prices. But National's Nicola Willis told Mike Hosking National would address the other two major factors – rates and electricity. She says they will push ahead with rates caps and continue consenting new energy projects. LISTEN ABOVE See omnystudio.com/listener for privacy information.
Auckland Business Chamber CEO Simon Bridges says politically, the Mood of the Boardroom shows a "tactical draw" between National and Labour. The survey of business leaders shows dissatisfaction and a desire for more investment in infrastructure, reform, leadership and talk of productivity, Bridges told Ryan Bridge. "Because you'd expect, and they are in their numbers, the Nats as a centre right incumbent government to be higher up, I would sort of call it a tactical draw between the Nats and Labour when you look at this closely." Unease about "the tail wagging the dog" with minor parties gaining more power in the future government is also present. "This is a very small sliver of business. It's the top end of town, let alone it being the electorate... "They've got a very clear view about what they like. They tend to like stability, certainty, an incumbent government over change that might be messy and uncertain." The former National Party leader said no result in the upcoming election would surprise him. LISTEN ABOVE See omnystudio.com/listener for privacy information.