#2525 - Nick Bostrom

[00:00] the job again experience during my day joe rogan podcast by night all day great to see you again yeah good to see you so um since the last time we talked we spent a lot of time where you were trying to explain to me simulation theory and why the probability of simulation theory [00:27] is more likely than it not being a simulation. [00:31] Yeah, it was what five years ago something? I think it was six. Five or six. I mean a lot of things happened in the world since then. Yes, yes, a lot of things. I mean, [00:43] For example, back then, I think we – did we even talk about AI? It probably came up a bit, but it feels like it was an era. It wasn't this thing looming over civilization, which is really kind of fascinating when you think about the fact that it's only been six years. And in six years, like what a massive jump in some new technology in our life. [01:06] Just sort of like the internet where it like crept up on us. We just accepted that it's a thing, but that this thing has gotten massively entangled in every aspect of society and every aspect of people's lives. [01:22] in a very short period of time. Yeah, I mean [01:25] Things are like so much is happening now that is kind of a full time job just to monitor the situation. Yeah. And you're one of the things that you're talking about is the positive aspects of it. Right. You're talking about.

[01:40] Like that this is probably going to be a net good for humanity. Hopefully. I mean, I think I take both cases seriously, the sort of the risk side and and the big unlock if we get things right. I feel like we have the potential like we're on a whitewater raft. We have the potential to get to our destination. [02:03] But we also have the potential to flip over and try to figure out how to get to shore and freezing cold water and sort of rebuild. Yeah, yeah, yeah. That seems about right. Yeah. And it seems like it's kind of speeding up a bit as well, right? If you're sort of in this field, there's like every few weeks there's a new model being released or something. Yes. I guess it's like if you imagine, I guess, commentating some fight, right? [02:33] First round is like normal speed. And then the second round is 2X. And then the third round 4X. And it's just like a whir of legs and arms. [02:42] That's a great way to describe it. Yeah. It's... [02:46] It's just so strange how quickly it snuck up on us in that there's two narratives that we hear. The one narrative is we're in real trouble and that this thing is going to take over every aspect of society and it's essentially going to be a superior life force, a superior intellect that exists amongst us that we created, and we don't think that's wise.

[03:16] It's going to be so much prosperity that no one is ever going to have to toil again. There will be no more third world countries. There will be no more poverty. Like we can eradicate poverty with the resources that we have on earth and we can change what it means to have to work. [03:33] provide yourself with food and housing. That's all going to be easy and free. And then everything else is going to be, you have to find a purpose in your life. My problem is, and I love Elon, but the people who have that perspective are all making money off AI. They are all invested like heavily, Mark Andreessen, all these people that have this rosy view of it. They're all invested [04:03] that is more of a true objective analyst of what's going on. When you have a positive aspect or a positive viewpoint of it, I get a little more excited. - Yeah, well, I mean, the truth is we don't know, right? How it will pan out. - Right, of course. [04:19] I mean I think there are these scenarios where we unlock this you know, enormous boost both to economic productivity but then, you know, across medicine entertainment, environment travel, all kinds of things and like a tsunami of wealth just kind of flows through and lifts all boats and then the idea of human work becomes an anachronism and where you have machines that can

[04:49] Do everything that we can do physically and mentally and do it much better and cheaper? And so I think in those scenarios where this really works it is [04:58] the transformation is a lot deeper. I mean, so that kind of layers to the onion. So like the most superficial level is, well, they automate your job. So what? [05:06] people are going to work. Well, that comes a little bit deeper. First, like the most superficial level is people just wonder where will I get the job if the robot replaces me. And then the superficial level of that conversation, I think is, well, you need to retrain workers so they can work in other fields. And maybe there needs to be, I don't know, employment insurance whilst they are being retrained or something like that. But once you think through where this ultimately leads, [05:36] to a good approximation. [05:39] with maybe the exception being where... [05:42] The consumer has a direct preference that the particular product or service be done by a human, like a priest, prostitute, and politician. [05:54] The three Ps. Those might survive. Well, politician, hopefully not. Hopefully AI can handle that all without any corruption. Yeah, but somebody's got to take credit for it. This is true. And code it. Yeah. [06:09] It's... [06:10] And so then you get to these like more profound questions, I think, about meaning and purpose and like what does a human life look like at technological maturity? Right. Also, what is experience? Does experience have to be measurable? Like do you have to touch it? Do you have to measure it and weigh it?

[06:31] is virtual experience still experience. [06:35] Like if you have a very full and enjoyable and fulfilling virtual life. [06:41] Is that enough? You know what I mean? Yeah. Like, do you have to do everything in the material world, or can you – [06:50] find happiness in a virtual world that doesn't currently exist, but we could clearly see the technology, if it expands, it's going to, there's going to be the possibility of experiencing a matrix type reality. Yeah, I think I see where you're going. Yeah. But yeah, and I mean, I think like different strokes for different folks, if you imagine a world where presumably these virtual [07:20] fascinating, but some people might just like the idea of, you know, Klein being the real Mount Everest rather than, you know, being most certainly, but however there could come a time where people are locked out of regular reality, like this is worst case scenario with AI, is that [07:38] You have your needs taken care of, but there is no purpose. [07:43] there's nothing to do. There's nothing to do because AI is taking care of all aspects of society. Other than recreation, there's nothing to do. And then this one recreation comes along that's not reality, but it's way more fulfilling and exciting than any other aspect of reality. And – [07:59] You know, an example that would just be in a very minor way.

[08:04] Your phone provides you with that sort of an escape, and it's not even that thrilling, and yet it's massively addictive. [08:11] People are on their phones six hours a day. If we come up with a virtual reality that's way more exciting than regular reality, [08:21] everyone's going to hop on in. [08:23] Yeah, I mean, I think to some extent this will... [08:28] maybe be the case whether you go the virtual path or the physical path. I mean, you could imagine a future where there are like resorts and people are sort of lying in beach chairs, sleeping drinks all day long. And like that's also a kind of checking out, right? Yeah, well, that, yeah. [08:43] That's not fulfilling. The thing about a virtual reality is – [08:48] You don't have to even live within our physics. I mean, you can fly. You can do anything. It's a larger space of possibilities. But either way, it could be like a passive existence in physical reality where you're like launching at the beach. Or it could be like a video game. Or in video games, like you could have like a really intense, every straining, every fiber to try to succeed in this virtual environment. Or like one where you're just kind of floating on some cloud in a drug-like state. But I don't think people are going to be interested in that. [09:18] Like what kind of video games are people attracted to? Because a video game is essentially – it's an approximation of that. You're just watching it on a screen, but you're sort of forgetting the fact that you're watching it on a screen. You're just concentrating on this 3D reality that you're running through with a machine gun or whatever you're doing, right? That's what people are interested in. They're not interested in these games where you just like float over the earth and like fly and just eat bananas. No, they're interested in thrilling things.

[09:48] in a virtual way, the amount of people that are going to just say this is better – [09:54] than regular boring life or we're a slave to a computer. Yeah, I mean there are both kinds. [10:01] This episode is brought to you by ZipRecruiter. We all like to find ways to save ourselves some time, like ordering out for dinner or meal prepping so you don't have to cook during the week. If you're a business owner looking to hire, a great time-saving hack is ZipRecruiter. Its newest feature lets you meet the most interested, qualified candidates first. Try it out free at ZipRecruiter.com slash Rogan. And it doesn't just let you meet candidates faster. [10:31] why they're interested in their own words. Save time and meet great candidates sooner with ZipRecruiter. Four out of five employers who post on ZipRecruiter get a quality candidate within the first day. Try it for free at ZipRecruiter.com slash Rogan. That's ZipRecruiter.com slash Rogan. Meet your match on ZipRecruiter. Okay, we're back. Where were we? [10:56] Oh, yeah, I was just saying that there are also these computer games that, like... [11:01] provide a more passive experience. Yeah. You're like not really doing much and it's more kind of zoning out. But most people aren't interested in those. Most people are interested in Call of Duty. They're interested in these wild first-person shooters where you're running down hallways and everything's exciting and thrilling, some sci-fi game or, you know...

[11:20] Half-Life, something like that. That's what people are interested in. Yeah. Well, I mean, I think there are some personality differences in what path you would take there. But either way, I think a lot of the choices that people would make currently – [11:34] depend on what they are sort of designed to respond to by feeling good about it. So like if you enjoy one thing, you choose that. If you enjoy another, now in this conversation, [11:45] a condition of technological maturity. If we imagine sort of a future civilization that has developed all possible technologies to their maximum. - Yeah. [11:56] Then amongst the affordances, amongst the things they could do is like they wouldn't just have control over the external environment, things around them, but also their own. [12:06] biology and brains. So if you were one of these people, and like maybe right now, the only way you get enjoyment is by getting like crazy drunk or doing sort of things you don't really approve of maybe, but that's actually what sort of lights your fire. Like you could... [12:24] You might not redesign yourself to take the same kind of pleasure, but in some kind of contemplating the beauty of the universe or like appreciating the goodness in the heart of others or like some more sort of noble aspiration. Like solving abstract mathematics rather than, you know, playing first person shooters. You would have a choice, like whether you would get your thrills from... [12:47] from one kind of activity or the other. Yeah, which brings us to the question, like, what does it mean to be human? Like, for people finding...

[12:55] beauty and joy in, you know, interesting and fascinating things and, you know, how to be a better person and all the different aspects of life. [13:06] human life that we think about when we think about people we think about noble aspects of humanity but Doesn't that have to exist? [13:15] in conjunction with the worst aspects of society for us to appreciate it? It seems like this is a part of the human condition, is that we have to have crime so that we can appreciate peace. We have to have war so we can appreciate peace. We have to have hate so we can appreciate love. And we're never without one or the other. They're always both together in constant conflict, and we're always nobly hoping that the good wins out over the bad. And this is part of the struggle of being a human being. [13:45] And if we just completely eliminate that struggle, we're going to have to find some – we're going to have to be a different thing because what we are is this very strange territorial primate who's endlessly curious. And this territorial primate is moving in a certain direction, and that direction seems to be a better society. It seems to be over time. [14:15] There's less violence. There's better medicine. There's better education. It's moving in a better and better and better direction. But it's in a struggle. That's part of why it moves. If all of a sudden there's no struggle and everyone is this wonderful, enlightened being, like what are we? Because we're going to be a different thing than what we are right now. And if you love music and if you love art and if you love novels and all these different things that come out of the human condition, those things come out of struggle.

[14:45] Those things come out of confusion and pain and heartbreak and love and joy and all that stuff all piled up together. Without that, like what are we? And so if we are moving in this direction technologically and we're not moving – [15:02] is fast biologically. [15:05] Do we merge? Because that seems to be what I think. I think that if this thing goes the way it continues to go, the bottleneck is going to be human biology. [15:19] embedded in [15:21] our efforts to make progress [15:24] So like there are all these kind of scientists and people working to develop better technologies and throughout the economy, you know, you're in some company, maybe you try to figure out how to make some process a little bit more efficient so you can serve customers better. And all of this is designed to solve problems, right? Like if you sort of extrapolate that. [15:42] to its logical ears. [15:43] end point right you would imagine we would have [15:46] perfect technology that can do everything, solve all the problems, [15:50] like presumably with AI and automation. - Right. - But then you end up in this condition where there is kind of nothing left for us to do, you might think. And so although it looks like we have these strong reasons to push forward in this direction, [16:03] If you actually look at the end point, if we succeeded to many people, it will look kind of unpalatable and like this kind of future where all the problems are solved. And nevertheless, that does seem to be the direction that we are headed in, probably the direction that we should be headed in. Yeah. Wouldn't that be a better goal if there was zero murder?

[16:26] Zero violence, zero crime, zero lying, zero corruption. Huh, huh, huh. [16:32] And human beings all worked in coordination with each other. Right. [16:36] Just like in a sense of unique syncing and harmony. Right. That would be way better. I think so, but it does... [16:45] Like if you actually stare at that situation, it does have these slightly unpalatable qualities. Because you might think it looks kind of bland then, right? Not necessarily though. It's just it looks bland if you think about what we're experiencing now. Like all the excitement and the weirdness of uncertainty and of not knowing and also the negative aspects of our life. [17:10] in conjunction with the positive aspects of our lives. We contrast the two of them, and you really appreciate good people after you're around a bunch of fucking assholes. You know, if – [17:19] But it doesn't have to be that way. This is just what we are dealing with. This is my position on work too because everybody in their head is like, what happens when AI takes all the jobs? [17:31] And I'm like, do you have to have a fucking job? [17:34] Isn't that a human invention? [17:36] Like, why do you have to have a job? It seems crazy that our main focus is on housing and food, right? [17:44] And like most people are basically just working for that. Most people that are most people that are struggling check to check, basically housing and food is their daily labor. Right. [17:55] If that's removed, wouldn't enough people figure out what to do with their time? Yeah, I think so. Like, yeah, there will be some transition and discomfort in change. But this is made up. But ultimately that is like, yeah. I mean, I think it's like – so slavery is really bad, right? Like wage slavery is a sort of slavery light in a sense. You have to sell a third of your waking day just to get money to pay for necessities.

[18:25] most money make the conditions for their employees as shitty as possible because it costs the least amount. [18:32] They don't think about it like, listen, maybe we make less money, but we have an awesome experience for everybody that works there. Nobody thinks like that. They all think like we have to make the most money because we have shareholders and we have a responsibility and these people need to make more money every quarter. Yeah, although sometimes you do need to compete for talent. That's true too, but only the top talent. [18:55] Some employees get treated well, right? What would you say? Jamie is a different kind of employee. Jamie is literally the co-person on the show. That's different. But some dude who's out there stocking boxes for Amazon, that's different than being the CEO. Yes. I mean I think ultimately that would be a huge liberation and kind of restore dignity. Like you should be in control of your own time. It's kind of almost the most – aside from your own body. [19:25] Like you're on time, you're on attention. We have to teach people discipline. They have to – that's the thing. It's like if you give people the opportunity to just do nothing, we do. We're primates. We'll find a way to – we're genetically sort of designed to conserve resources. You don't want to overuse resources for no reason, do a bunch of shit because you won't survive. You don't have enough food. So we're sort of designed to like know how to be lazy. And so we're going to have to find things that give us purpose or we'll get depressed.

[19:55] This episode is brought to you by Visible. It's officially summer, folks, and boy, what a summer we are having. It feels like one of those good old-fashioned summers where you just want to enjoy every moment and keep up with the action. Your best companion this summer, Visible Wireless. Visible gives you unlimited 5G data and unlimited hotspot powered by Verizon for just $25 a month. So you can make this the summer of fun and savings. Visible is more than just a wireless plan. [20:25] It's unlimited wireless designed to always keep you connected with no contract to tie you down. Switch today at Visible.com, plan start at $25 a month, or get our premium Visible Plus Pro plan and save $10 on your first month when you use promo code ROGAN. Terms apply. See Visible.com for plan features and network management details. [20:55] that you wouldn't have to work for a living. It was kind of an unfortunate necessity for many people that they had to work for their daily bread, right? But the ideal way of being human was that you just had money. And then you spent your time doing other things. Maybe you were involved in politics or maybe you had an art collection or you did your gardening or entertained friends. That's the best aspect of it. But when you think about British aristocracy, don't you think about fuck-ups? I do.

[21:25] drunk, they drive their Ferrari into a lake and they accidentally drown their friend and then they pay off the cops and they get away with it. That's what I think of. Well, yeah, there's that. But there were also these kind of, I don't know, amateur scientists or eccentrics that have their weird thing and it can't have it. So I guess it brings out like the more freedoms of what it has, the more they can reveal their true nature. Well, it also depends on how they're raised, right? Who are their parents? What values did they instill in them when they were young? [21:55] Did they explain the value of hard work, that it's actually good for you? And that if you just find something you really want to do and you concentrate on it and really work hard at it, it's actually very fulfilling. Yeah. So you could imagine, for example, the education system in this world where we no longer need to work. Right. Presumably it would need to be redesigned from scratch basically because right now it's a kind of machine, right? So you take children coming in kind of on a conveyor belt. Yeah. [22:25] here's this assignment, here's your grid. And then they come out with a quality label attached. Yeah. And then they are meant to be productive workers that you could put in an office and they will... [22:35] So this kind of is an unfortunate fact about the current condition that the world needs a lot of these office workers that can do all these tasks in the economy. So like it's we have to have until it doesn't until it doesn't. And at that point, it would be absurd to keep. [22:51] doing this, right? - Right. - Then you could imagine changing education so that it would be much more about like, how can you

[22:58] Learn to... [23:00] live well a life of leisure. Like you met and... You're right. Stay stimulated. Cultivating the art of conversation, appreciation for art and music, hobbies, physical wellness, like nature, the ability to [23:16] set your own goals and take your own initiatives to, you know, to develop true friendships with people. Like these are things are not taught in school, but you could imagine that being the curriculum, like spirituality, like all kinds of things that would then equip people to sort of use their freedom, their wealth, their free time for, you know, some kind of, [23:39] you know, actually meaningful and beautiful activities. For sure, preemptively, we should be kind of teaching people like that now. Like if young people are coming up now, the world that they're, if you're in first grade right now, 12 years from now, when you're graduating from high school, like the world is a totally different place. And this idea that being a productive worker who could sit still for eight hours a day is, that's the best way we should teach kids. That seems crazy. [24:09] like seeing the world that you're describing and that most people seem to think is oncoming. [24:17] The timeline is a little bit uncertain. So you don't want to end up in a situation, right, where you sort of now you find yourself, you're 20 years old. It's time to go out in the world. You have no skills. The AI revolution hasn't yet quite happened. It's kind of 50 years old. Well, isn't that a liberal arts degree anyway? Yeah. How is that going? There's a lot of silly degrees that people get right now that are useless anyway, and they –

[24:37] spend a tremendous amount of time working on getting it. And then when they get it out, there's no jobs for that degree. Yeah. So they were a little bit too early maybe in hopping on this. Maybe. Or, you know, there's also... [24:50] traps, there's things that you can get really excited about and do. Like if you want to play professional bowling, you're [24:57] You want to bowl professionally. Well, the amount of money you're going to make is very limited because the best bowler in the world – like what does the best professional bowler in the world make? [25:07] Let's guess. [25:08] I say $100,000. Yes, they're professional bowlers. We used to know a guy, Ari's friends with a guy, Tommy. God, I can't remember his last name. Do you remember his last name, Jamie? Jamie. [25:20] He was a professional bowler, but he used to come to the comedy store and hang out. Really nice guy and just loved comedy. I don't know. Maybe the very best one makes a... [25:28] pretty decent living and then like the third best like it's just i bet barely barely i bet the best guy probably makes maybe 100 grand or 200 grand at the most and that means like the 30th guy in the world is fucked yeah there you have to have another job to pursue that well but then in in this what does he make [25:47] Oh, $200,000. Oh, $400,000 in big season, $450,000, including prize money and sponsorships. [25:54] E.J. Tackett made $437,540 for the 2025 season, with several others between $190,000 and $270,000 in prize money alone. Top tier.

[26:11] So the top tier guys make a good living. [26:14] But how many guys are there? [26:15] You know what I'm saying? It's probably like a very thin sort of peak. So if you choose to go into bowling, it's not like choosing to go into – there's a lot of other jobs. Like even if you don't want to be a basketball player, they're making a lot of money. If you're a really good basketball player and you pursue that, like you could get very, very rich. But how many of them get rich? Most don't. Yeah. But then like – so maybe the money factor would kind of drop out of the picture. Yes. And it would just be what you're interested in. And then maybe even it would – [26:45] I mean, the opportunities are endless. And what other people are interested in to some extent because I think part of what people are competing for is also prestige and status. Yes. That would be a thing, right? Like status would be more important. It's like who's the best at this thing? Like a bunch of friends who play golf. Like Jamie plays golf all the time. Golfers are all like comparing each other's scores and they're playing. They're competing in this game. And they think about that more than they think about work. Like people who love golf, they fucking hate work. [27:15] I'll say that about comedy. Back when I first started, one of the things that I noticed is the guys who really got into golf, their careers kind of stalled because they were more excited about playing golf than they were about writing jokes and going on the road. I was like – [27:29] Okay. So if the average person doesn't need food and housing anymore from labor, if that's gone and now you just get it, and so now you could just go do things. [27:40] We just have to teach people to be excited about stuff. We have to teach people the value of curiosity and finding things that are interesting to you. And then the value of just education for the sake of learning things because it's interesting. Just pure satisfaction of curiosity, which is a beautiful thing. Like that would be great for everybody. If we – instead of learning things because your teacher tells you you have to learn it, well, there's always going to be people that just naturally gravitate towards mathematics.

[28:10] fascinated by mathematics. And there's always going to be people that are just naturally gravitating towards history. They're like, how did we get here? What is this? How did it start? Who wrote that? Who's the first guy to figure that out? And that that's you're going to naturally go towards that and just be educated for the sake of satisfying your curiosity. And maybe we'll have a more balanced society because people will actually be able to just pursue their interests. We have to teach them to do it. That's what I think is going to be really important about young [28:40] Teach them to actually pursue their curiosity rather than just squash any desires they have for novelty and for interesting things and just be able to work all day doing something you hate because that's what it means to be an adult. Yeah, I think cultivating that would be the way forward in this case. And it seems very possible. It's doable. We know curious people. Yeah, at least to some extent. [29:10] probably also some differences in how different folks are wired like some people get bored very easily others find everything fascinating yes people are like just naturally depressed or have low mood others are sort of yeah but those people that are naturally depressed how much would their life change if they were coached at a very early age the value of exercise and if they started running if they started doing yoga and they started feeling much better and they weren't naturally depressed anymore and then you change their diet and you start adding in vitamins and

[29:40] foods and all of a sudden they feel better because the concept even of depression, like what does that mean? Well, it varies so wildly. And when you're talking about people that aren't taking care of their bodies and that's not thought of as a primary factor in why depression exists in the first place, well, this is a very unscientific approach then because we know that it has a profound impact on human beings. So we're pretending that there's like this one size fit all. [30:10] to be a human being. Chase your curiosities, but also, this is why you're not feeling well, and that if you just develop this ability to get some momentum going and just show up at that yoga class every day, you'll feel way better, and you don't have to take a pill. I think, yeah, [30:28] many could be a lot improved if they were taught to sort of take care of their basic physiology. Everyone can be improved. 100% of them can be improved. It's a part of being a human being. Yeah, yeah. Well, I mean, I don't know. I think there is also some people who might just have some... [30:44] imbalance of some neurotransmitter because of some genetic... 100%. That's possible too. Or maybe they don't even have the energy to engage in these activities that would make them feel better. But then I think that would be much better medicines for helping... That's where Adderall comes in. Let's go. Give them some Adderall. Get them fired up. Get them to start running. [31:05] I'm kidding. [31:07] But what we're saying is all doable. We're not asking people to breathe underwater, right? What we're saying is that there are people on earth that live like that. And this idea that living just for a paycheck so that you can cover your food and your housing, which we've always thought of as being –

[31:27] Just... [31:28] an undeniable part of being a person, an adult. This is what you have to do. You have to pay for your food, pay for your housing. But if we do get to a point where – [31:36] The structure of society is now run by hyper-intelligent artificial intelligence. You wouldn't need most jobs that people have to do that suck. [31:47] And in order to get our society to this point, if you wanted an iPhone, you needed some people that were in a factory somewhere putting it all together. You needed someone who's designing it. You needed someone who's sitting there in the office trying to figure out how to market it. You needed all those jobs. But when we don't need all those jobs anymore, then things are going to be very interesting. And that's – [32:09] What I'm saying, we need to become a different thing. [32:13] It's kind of true and kind of not. Like you can be a human being and live in that world. But we're going to have to re-educate people how to be a human being. It's going to – this – [32:24] Our education system specifically in this country is just designed to make factory workers specifically. Like there's a real history of it. Like we know why they made it this way. They made it this way and they got people in really early so they could get people set up for jobs. They just want people to work. [32:44] ill-suited for really thriving in an environment of abundance and for enjoying life. [32:53] Both at deep biological timescales and [32:56] during the lifespan of a single individual, there are all these pressures, necessities, that kind of force us to become a certain type. So I mean, we talked about the education system, training people to be the kinds who can sit at the desk all day long and perform tasks. - Yeah. - Biologically, we've,

[33:14] evolved drives to be lazy, to conserve energy, to eat as much as we can. And now in the modern environment where there are fridges everywhere, like it causes problems metabolically. And [33:28] in terms of enjoying life for many people, there is this hedonic treadmill, right? Like, so you achieve something [33:34] some improvement, there's a spike of happiness, and then you sort of... [33:38] go back to baseline very quickly. You start to take for granted all the blessings of life, which makes it very difficult to actually achieve a state of, um, [33:48] permanent happiness and felicity. [33:52] And it's kind of been necessary because there needed to be this [33:57] motivation to keep striving for the next thing. Now, once you've actually achieved all the things, then maybe that becomes kind of dysfunctional, right? Like why keep wanting to strive for the next thing if all the things have already been achieved, at least in a certain domain. And so I think as we move deeper into this [34:16] hypothesized future, right? [34:19] where we get all these magical technologies, then at some point, probably some transformation [34:25] of human nature would have to go along with that. First, maybe cultural changes to sort of equip people for a life of leisure. Yeah. And then ultimately, maybe more profound changes to our very biology. And so that imagine if you solve aging and you can now live [34:42] for thousands of years, right? Like maybe the way our memory work is not really set up for them. Maybe we would just go stale.

[34:48] after 200 years with our current brains like we just get stuck in a rot and [34:52] We don't know because we haven't lived for 200 years, but that could easily turn out to be the case with a human. So then maybe at that point, you would need to sort of do something to sort of add more cognitive resources, more flexibility, do some sort of psychedelic reset or something to sort of keep the flexibility going for longer periods of time. [35:22] and specifically with psilocybin and DMT and probably ibogaine as well. Like if we wanted chemical tools to navigate a new reality, those are probably the best ones that we have available and they're all illegal. And I think you're right. I think if we're going to be able to navigate this correctly, we have to kind of change the way we interact with each other, what it means to be a person. [35:48] But I wonder how much of the conflict that we have is a direct result of this inherent struggle that so many people have. [35:57] I mean there's a direct correlation between extreme poverty and extreme crime, specifically in this country. If you look at the areas of extreme poverty in this country, they're also the areas with extreme crime. And I wonder how much of that would be completely alleviated with a complete lack of poverty. [36:15] We've always assumed that – [36:17] If you're going to have –

[36:21] a functioning society that you're going to have slums. [36:24] Why? Like, why? They don't serve any function. It's not a good thing. Well, it's because some people are always going to be making bad decisions and some people are always going to be going down the wrong road and crime and this and that. Right. But they're still just humans and some humans don't do that. So wouldn't it be better to figure out a way to not have humans ever do that anymore? And it seems like a way to – there seems like a way to do that. [36:54] never engage in crime. Why do some people live these really fulfilling and interesting lives? Probably because they were exposed to it when they were really young, probably because they weren't exposed to like very bad environments and very bad crime and very bad poverty. And how much of that would change if there was no more poverty? It sounds like such a little fairy tale childlike, oh, one day we'll have no poverty. But [37:19] That's doable. If everyone is alive, right, okay, that's alive right now is not starving to death. That means we figured out a way to at least – the very least get you resources so you can feed yourself, right? No matter how – [37:33] dysfunctional things are. [37:35] All it has to do is get ramped up a few steps. I am. [37:39] And now you have no one ever worried about being fed, no one ever worrying about not having a place to sleep. And then you have to find purpose. [37:47] But it seems like there's a lot of people that find purpose without having a financial price tag attached to it just by what we're talking about with golf or –

[37:57] You could be like really into writing books. You could write books all day long and people are always going to want human created fiction. People are always going to be interested in the way other people think about things. [38:08] You'll find – there's plenty of things to do. There's plenty of games to play and plenty of skills to learn. The idea that the only reason why we work is to eat and to not get rained on seems nuts. Yeah, I – [38:24] It is, I guess, to some extent, an open question to what extent people will always want. [38:29] to read human fiction or to prefer the human generated outputs. It might be just because current AI generated writing is kind of lacking in various ways. It's often slop and boring. But if it became really good, then it's not. [38:49] You know, maybe it would just be, like, much more fun to... [38:53] Like if a movie made by an AI might just be so much better and richer and the lighting is perfect and the dialogue is sharp and it's more funny and deep and it really touches it. Then you go and watch this human-produced thing and it's going like... [39:06] Like most people don't go and watch sort of film school students' productions. Right, but then there's that movie obsession. Wasn't that movie made like very inexpensively? [39:16] And it's a huge hit, right? [39:18] And that's part of the thrill of it, is that this guy who was like a YouTuber, right? [39:23] who created this film, and now this movie is a giant hit, and everybody is super excited about it. Now, if that same movie was made by AI, I wonder if it would have that impact because it doesn't have the human connection.

[39:35] So that's possible that we keep... [39:39] getting interested in these things because we sort of are really entangled with the human world much more than we are with the world of AIs. And, I mean, for the same reason you might be more interested in if, like, your brother or friend did something, you might think, oh, this is interesting. I want to check that out. Of course. Whereas it was some random dude who made something slightly better. It's like doesn't mean – Right. When you go to see your nephew play baseball, you wouldn't really go see little kids baseball. No, no. Just randomly, right? [40:09] on first base, like, way to go, Bobby! So I think these kind of... [40:13] social entanglements that we have will be a big part of what gives... [40:18] structure to lives in this condition where the external constraints are removed. I think that there is a great value for the human mind, for whatever reason, in getting better at things and learning things. And I think that if we could instill that in people at a young age, I think we're [40:39] It would be fairly easy to get people to pursue that path. But we have to completely revamp our education system, which should have been done a long time ago anyway. The idea of having unenthusiastic, underpaid people, being any percentage of what children encounter for most of the day when they're in their most formative period is insane.

[41:09] to learn things and that was exciting to get better at things and about how anybody can get better at things. I mean, I hated every day of school. I hated it. I hated it. I didn't even know that I was interested in learning anything until I got out of school. [41:22] The same for me. I mean, I only discovered – like I also hated books and everything like that for my first 15 years of life because I associated that with school. That was the kind of school thing. So it's only then randomly one day I happened to walk into the local library for no reason and pulled out a book here and there. And then I discovered that there was this whole world of science and idea and literature and all of that, like very, very different from what we were doing in school. And then that kind of opened the gates for me to this. [41:52] you would have had if you had like... [41:54] a different kind of education. Yeah, I am. With super enthusiastic people who really love teaching children, are really good at it, and that we reward them, and that it becomes a very prestigious position to be in, rather than what it is now. If you talk to some guy, and then he goes, oh, I'm a high school teacher, you're like, oh, poor bastard. How does he feed himself? That's immediately what I think. Like, good for you that you're doing that, but also, I guarantee you could probably be making a lot more money and be happier doing another job. Yeah. [42:24] And so that's a terrible way to start life off. And if we just revamp that, then you have a bunch of people that are interacting with life in a very different way. And instead of being thought, I have to get a job someday, I have to get a job after school anyway to help my mom pay the bills because we're struggling. So I got to contribute to the household, even though I'm 16, I can't hang out with friends. If all that stuff's out the window. And now it's like, no, what we need to find out is what is exciting for you. Like what?

[42:53] What excites your mind, your specific personality? What is it about life that's interesting? And let's expose you to a bunch of different things that are exciting and interesting that other people find value in. And let's find out which way you gravitate because maybe you gravitate towards chess or maybe you gravitate towards something completely different. Maybe you're just really in a painting. Maybe painting just lights you up and you like look at a canvas and you just start fucking around with it and that's your thing. There's a lot of people in this world that find that. [43:23] They have to find it themselves, unfortunately. Yeah, yeah. I don't... [43:27] I think that's kind of inspiring. But I think... [43:33] the train doesn't stop there. So if we think even more further into this kind of [43:38] Thank you. [43:39] condition of technological maturity i think in in addition to sort of freeing up people you know like making it easy to produce the food and the houses and like cars that go faster and don't pollute and all of this stuff like if you think through what maybe a technologically mature civilization could do like so a lot of these um like learning for example is something that gives people purpose but maybe you would have the ability to sort of download knowledge um at the [44:09] learn advanced mathematics now you have to study year like for years right do [44:13] - Right. [44:14] like books with exercises and apply yourself and then eventually you sort of unlock things [44:19] Yeah. But like maybe at technological maturity, that would be like, okay, I understand mathematics. Okay, I'm going to press that button. Boom. Right. Like the matrix. I know Kung Fu. Remember? Yeah. So if you had like, you know, you may be some kind of nanobots that could infiltrate your brain.

[44:36] change the synapses in just such a way that you're not the same as you were before, except you have these concepts from abstract mathematics, you know, after 20 minutes or something like that. Some superintelligence works out how to change your synapses to this new condition. And if you were... [44:53] inclined to do various things because they give you joy and pleasure. You could also think, well, [44:59] I could do those things and get the joy and pleasure, or I could just push this button that activates the same joy and pleasure. So there would be these shortcuts to everything. [45:08] and it looks like you would have a post-instrumental condition where that would... [45:14] at least at first time, it seemed to be no reason to do anything for the sake of achieving something else because there would always be this like shorter pass to that other thing that [45:23] pressing the button. Yeah. And so that's a kind of deeper form of redundancy. It seems that all forms of human instrumental effort. [45:31] would become unmotivated. [45:33] You could still... [45:36] you know, go to the gym every day and sweat for an hour. [45:41] Or you could just take the pill that induces exactly the same physiological effects and the mental effects like the calm or whatever you feel after. [45:51] Or maybe even better, genetically engineer someone. Or genetically, yeah. So they don't have to take a pill. Yeah, yeah. [45:56] um... [45:57] And so then I think you're getting more into these fundamental questions of value, philosophical questions about what ultimately is it that makes a life good? Yeah.

[46:08] And what makes life good for a primate? [46:10] because that's what we are, or a primitive version of primates. I think we're going to become something different. And if we do it through technologically-induced evolution or biologically-induced evolution, we're still moving in a general direction. Just biologically, if you look at ancient hominids and look at us, we're clearly very different. So we're moving in a general direction anyway. And when you look at the greys, like these prototypical aliens that everybody sees, the big heads and the skinny bodies, what do they look like? [46:40] We keep going. That's what we're going to look like. We're going to be these genderless, sexless, muscleless, little skinny things with giant heads that communicate telepathically, which is probably where we're moving. And if we're moving that way, maybe we think of them as being an alien from another – [46:56] Maybe that's just the general direction that primates go once they figure out technology. They eventually realize, well, first of all, the idea of having to have a male and a female is kind of crazy. Why? [47:11] Well, because we have to reproduce. Well, what if we don't reproduce like that anymore? What if all reproduction is done through engineering and there's no more sex, so there's no more lust, so there's no more jealousy, so there's no more ego? There's no more anything. And then there's a hive mind because technology has advanced to the point where the reason why they have these big giant heads, they're essentially locked into everybody all around them all the time. Yeah, well, is that what we would want though? Not us, but I don't think we'll be us anymore.

[47:41] I think this – like if you went to chimps and you said, hey, dude, one day you're going to be on a plane eating peanuts, flying over the ocean. They'd be like, fuck that. I don't want to do that. I'll just stay here where the fruit is. You guys are nuts. Like why do you want to do that? And you'll be addicted to Kratom and you'll be watching YouTube all day long. Like, no, I don't want to do that. That doesn't sound fun. We do have a lot of bananas, though. Maybe. Or you have to get killed by the other chimpanzees that want the bananas. They have chimp wars. [48:11] come in and kill each other. So I think [48:14] We're going to be something different, and I think that's inevitable anyway. If you believe in evolution, it's inevitable anyway that we become something different. We're not a finished product. Yeah. [48:23] Yeah, no, I think that is true. We are not the finished product. Now, the question is the thing that we become or that. [48:30] becomes is that [48:33] Um... [48:34] a result of [48:36] us choosing how we want to be or is it just these kind of impersonal forces like evolution selecting certain types that might end up leading to outcomes that we actually don't want? And so I guess the hope is that we would be able to develop along a path that we [48:54] preserves and develops [48:56] the things that we actually value about being human, maybe amplifies them, then maybe adds other things. So there are many different possible things you could develop in the future, but that we sort of select those that actually make us better rather than just randomly different.

[49:13] that we can sort of grow into our ideals [49:16] But it's so funny how much value we put in human condition. [49:21] how much value we put in meaning because it is valuable to us. [49:26] When you think about like what that means in terms of the amount of energy it produces, the amount of impact that it has versus the structure of the universe itself versus black holes and stellar nurseries and things that are just infinitely more spectacular than the human condition. But to us, it's like, oh, what is meaning? What is meaning? What is meaning to me? [49:56] sort of tromping around through this weird world where eventually you're going to die and you're going to leave your mark and maybe reincarnation's real, maybe the afterlife's real, nobody fucking knows. So you've got to have meaning. [50:07] But the universe itself, like a human only lives 100 years. How much meaning can you have in 100 years? Yeah, I think we kind of die prematurely. I mean, we think of us as like first you become stronger and more capable for 15, 20 years, right? [50:24] And then biologically, you stagnate. And then maybe you keep accumulating sort of knowledge and skills for another few decades. But just as you have started to acquire like a modicum of wisdom, like your brain starts to rot. [50:37] And then everything is erased either by Alzheimer's or death. That's all gone, the whole lifetime of experience. There's your meaning. All the memories and the hard-earned lessons. So that seems kind of sad and probably not the best way for things to be. I think we would want to extend the human lifespan so you could continue to grow up. Not just kind of reach 20 and then plateau there for the rest of your life.

[51:07] Like at 80, you were as much stronger and like be able to understand more and move better and like just have the same capacity increase as you had like between like zero and 20 and you just kept going. Well, these life extension scientists that are working on these things, guys like David Sinclair, like they believe that that's not just a possibility but an inevitability. Yeah, well, so that's more like, I guess, preserving, preventing or delaying that decay. [51:35] Sure. Which is already very good. Not reversing. They're talking about reversing. Yeah, yeah. But then you would go back to being 20, right? In terms of your physiognom. Biologically, yeah. But then you have engineering. If you have very long time spans, then you might at some point want to continue to grow. Yeah. Like you might not want to just be stuck at a 20-year-old human for 10,000 years. Of course. Like maybe eventually you would want to become… [51:58] So like slightly bigger in terms of your ability to engage with the world. Of course. And if they can figure out how to make people – well, there's already genetic engineering that's being done in terms of increasing potential intelligence and IQ. They're already doing that. [52:28] inoculated to HIV, but it also... [52:32] at the same time gave them a far increased potential IQ. It remains to be seen whether or not it actually works as these guys, you know, become like 20 and 30 and we start putting them in chess tournaments and see if it really did make them smarter. But if the possibility of that is already being studied, they already understand the differences between like what

[52:56] What is what's possible what we understand and that increases every day they understand more every day more as possible every day if that just keeps going well you have a different version of what the human consciousness is you have a different version of the human mind. Yeah, I think the way AI is going I think that's the train that's going to reach the destination faster. Oh, yeah. And then. [53:21] Once you do have super intelligence in machine substrate, then that can then unlock all kinds of technologies, including these biological technologies or nanotechnology or a host of other things that could then sort of bring us up if that's how we decide to go. Yeah. And uplift us into different – like either biologically enhancing us or like uploading us into – But again, it's that whitewater river raft – [53:47] It's like we're going down this whitewater and we might make it out of this or it might crash. [53:54] You know, one of the weird things about – I'm very fascinated with – [54:00] ancient civilizations. And one of the weirdest things about ancient civilizations is when you go really, really far back, a bunch of them have these depictions of kings that ruled for thousands of years. And it's very strange stuff because the Egyptians and ancient Sumer, there's a bunch of different depictions of these kings that lived lives that were way longer than biological humans lived. And then

[54:30] There's the flood myth or the flood story. And then after the flood, biological life decreases pretty radically. And it seems to get back to like a normal version of what we assume now, which is like 100 years. And one of the weird questions that these – [54:47] alternative ancient historians have is are we missing – [54:51] the possibility that there was a hyper-advanced civilization. [54:56] that existed tens of thousands of years ago, and that what we're seeing now is not a linear progression from caveman to human being with an iPhone in 2026. That along the way, there might have been a very high level of sophistication, and the evidence of that might be the pyramids and some of these other ancient structures that are mind-boggling. [55:26] that are in the ceiling, perfectly cut granite, where it looks like they have diamond drill holes in them. It seems like there's some lost technology. And every society has this flood myth. And every society that has this flood myth [55:40] Especially these ones that were very advanced somehow or another like ancient Egypt. They have these depictions of people that lived way longer than people live today. And I wonder if human beings one day will realize like, oh, if you keep going long enough, 100 years is silly. Like people can – they'll just figure out what it is that makes people age and die, fix that, and you'll live an insanely long time.

[56:10] for reason and logic increases. Their capacity for intelligence increases. Then you have these insane technologies that were required to do things like build the pyramids. And that might be what happens. That might be a natural course of progression. The first thing we realize is 100 years is not enough because I'm 58. I'm a moron still. Like, why am I learning every day? How come I'm getting better at being a person every day? Well, I should have figured this out already. But you don't ever have time to figure it out. You don't have time. [56:40] I only get one run like I mean right you take it one trial run first right on life right like I can I I kind of know a little bit how it works now now let's do it for real that's the car is kind of scary I mean it's a bit crazy that [56:53] each of us is put in charge of a whole human life. Right. Like, are we really, like, reach the level of... Like, if you want to run an airplane, you have to go through these certifications and stuff, right? And then maybe you can be a pilot. But, like, for a human life, it's like every single person... Okay, you have... [57:08] 100% dictatorial control of this person. [57:12] They're completely hostage here. [57:14] whims. And then that's like us. But that's the way it is. But yeah, having the ability to kind of try different things and maybe do like have a kind of [57:26] opportunity to do a do-over if like if the life turned out to be like every 50 years or something you have a chance to kind of um or maybe not even a do-over but it's the same life but you can do whatever you want and you don't have to be constrained to this idea that you only have a certain amount of time you want to retire by your 65 we might look at that as being like a complete one of the first completely archaic notions and the reason why people got it all wrong like well

[57:56] lived way longer. And if you think about how much smarter, how old are you? [58:00] 53. Okay. Now, the way you said that was sad. I'm older than you, dog. Don't worry about it. But if you think about how much smarter you are at 53 than you were at 13. [58:12] Now imagine how much smarter you'll be at 353. [58:17] or 1,053. And if that actually becomes possible, if a person can live to be 100 years, why can't they live to be 4,000 years? If they can figure out what makes people age, if we really can genetically engineer human beings, that's not, again, that's not breathing underwater, that's just extending life. And if you extend life and extend intellectual capacity and your ability to learn and grow, holy shit, you're dealing with a 2,000-year-old person, that's a completely [58:47] And that could be real someday. And that might be also one of the things that comes along with this new understanding of just life in general with super intelligent AI. Yeah, no, for sure. [59:01] fixing aging and reversing it would be amongst things that a technologically mature civilization is. [59:08] would be able to do. We just have to get people out of the idea that it's vain. [59:12] Why do you want to live forever? Come on. That's what they say now, right? Yeah. But when somebody actually has the therapy and you could – like I'll hear – you could either continue to get worse kidneys and more painful knees and wrinkles and you start to forget things. Or you can take this –

[59:29] this rejuvenation serum that everybody else is using and they are like full of energy and running around and doing stuff like [59:35] Do you still think it's vain? So I think a lot of this is Stockholm syndrome that you are kind of... [59:42] faced with the inevitability of aging and decay and have been for thousands of years. So you develop a kind of romantic scaffold that reconciles you to the inevitable. I like it. Stockholm syndrome. Which is maybe... [59:55] adaptive, but only up until the point where there actually might be something you could do to escape. [1:00:00] then if you're too stuck in this mindset, it might sort of prevent you from taking advantage to regain your freedom. Right. And I think we are now at that stage. We don't yet have the therapies, but… [1:00:12] there are certainly like investments and research and stuff we could do that would hasten the arrival. And I think, [1:00:18] In fact, we should have started on a big sort of [1:00:21] Manhattan product to defeat aging. [1:00:24] decades ago and maybe we would have better therapies by now then but for a long time aging was not seen as a problem only the specific diseases that are the result of the aging yeah they got all this funding right like so you have like huge research products for alzheimer's and for heart disease and for cancer but if you look at the the main common cause of most human disease it's senescence like you're much more likely to get any of these when you're 80 than

[1:00:54] the arteries clog up, your DNA mutates, your methylation pattern gets scrambled. Your brain shrubs. So rather than just trying to put out the fires one by one after they occur, like at some point you just need to sort of... [1:01:07] try to prevent, like maybe you need a bit of rain to sort of [1:01:12] prevent that. It's kind of amazing how much we've accomplished as a society, as civilization, when you think about the fact that people only live to be 100. [1:01:20] Kind of incredible. [1:01:23] Kind of incredible. Got rockets and satellites and cell phones and people only live. So it's like everybody has to build on everybody else's intelligence and everybody else's understanding of the world and develop new things. And everybody has to learn those new things. And each brain is so small that it can only learn a small little specialty. And you need millions of them. Yeah. Although it's not entirely clear with. [1:01:46] with humans. [1:01:48] If we had been much more long-lived, like the way we currently are configured, maybe we would just have gotten too stuck in our ways and sort of stagnated into some sort of ultra-conservative society where nobody's allowed to do anything different. Or the old geezers that have their way of doing things just remain forever in charge. Yeah. So... [1:02:08] I mean, we just don't know, right, whether that would have slowed progress. [1:02:14] or accelerate it. [1:02:16] um [1:02:17] But it is still amazing. Like it's a long path from sort of running around in the forest to sort of these – look at these –

[1:02:26] like advanced chip technologies and the whole global supply chain. Or like thousands of people are working to develop just one little tool that then feeds into the ability to make another tool that eventually makes these leading edge AI chips. Where they're layering things four atoms deep. Yeah, yeah, yeah. It's remarkable. It's incredible. And humans only live to be 100. Imagine what we could accomplish if we lived to be 3000. [1:02:54] or if we were just a little bit [1:02:57] better at these things. Because I think we are sort of the stupidest possible species that are capable of developing advanced technology. Because as soon as we evolved to reach that level, we started developing it, right? And then... [1:03:11] So that's where we are, like the dumbest possible. I just barely can do this. Right. Barely can do this while we're blowing each other up. Yeah. [1:03:17] Yeah, at the same time, all over the world, blowing up people. Yeah, but maybe a lot of that frantic, stupid, illogical behavior is because we're so finite. [1:03:31] Like we're in a rush that we realize we only have a certain amount of time to get things done. And so this sort of accentuates – [1:03:38] the [1:03:39] the desire to control resources and to cement your immortality and to do these things that people love to do, put their name on buildings. There's a thing that people like to do that's almost like cementing – [1:03:50] Their immortality. [1:03:51] Maybe there'll be less of a desire to do that if people lived way longer. [1:03:56] And then you would have to assume...

[1:03:58] If you can – [1:03:59] engineer humans to live longer you could probably engineer a lot of stupidity out of us [1:04:05] They can just find out why people behave. Like what if you could just eliminate lying? What if there's like a genetic – [1:04:14] solution to lying. Well, or just a really good lie detector. [1:04:18] Yeah, but I mean what if you can genetically engineer out the desire to lie? [1:04:24] Well... [1:04:26] I mean... [1:04:27] If you had a perfect lie detector, there would be no point in lying because people would just see it immediately, right? Or if you could read each other's minds, it would be fruitless. Yeah, and so that might be closer. I mean who knows what you could do if you sort of had like a powerful AI system trained to detect micro-expression. We just all have to get the big gray heads with the black eyes like the aliens have. Then you can just read each other's minds. You want one of those? [1:04:57] stuff but he was one of the guys that um [1:05:00] Was a whistleblower that said he back-engineered crafts for the United States government in the 1980s. They don't have any controls in these crafts, supposedly, and that they're all powered by the minds of the beings that are running it. And that they have – it's almost like these crafts are alive and they have some sort of a syncing with the thing. That instead of like pressing buttons and working a joystick, they just sync with. [1:05:23] with this creation and then they can propel it with their minds. So their minds become the computer that

[1:05:31] that moves this thing around. I'm a bit skeptical. I haven't looked into that. You should be. Sounds crazy. How could you not be? Anybody who's not skeptical sounds crazy. But if you think about where we are now and where we're going to be, the possibility of that being our future is pretty likely. Right. So I think like that kind of. [1:05:52] possibility of engineering something like that with sufficiently advanced technology would be there like all kinds of stuff yeah basically everything that doesn't violate some law of physics um to a first approximation you would be able to construct even when we say that it's like our current understanding of physics it's not the understanding of physics a society that's a million years older than ours you know because that's i mean that's a blink in the eye of the [1:06:22] from the Big Bang. Like a million years is nothing. [1:06:24] Yeah, but even with our current understanding of what physics permits, that's still an enormous space of... [1:06:34] uh, [1:06:35] designs and types of life and being that [1:06:38] you could imagine instantiating. You could have, you know, like a Dyson sphere. You've heard of this concept, right? It's basically like using the output of the sun for energy generation. And it could have like, you surround the sun by solar panels that then powers a civilization infrastructure, maybe like a computer, like imagine the... [1:06:58] Like already we have like AI computes growing at like 240% per year or something, right? Right. With this. And then like imagine the kind of mind that could run.

[1:07:09] on that sized computer. [1:07:11] So people [1:07:14] I'm wondering whether, like, [1:07:16] Could machines be conscious and like discussing that right now? I think like maybe the more pertinent quantum is like, are we really conscious? I think barely. So you're diving on. [1:07:30] like the road for two and a half hours, a motorist, right? And like driving past thousands of people and homes and like mothers with their strollers and then [1:07:40] Thank you. [1:07:41] After just... [1:07:43] Does he remember any of that? Like, was he even really aware of any of this while he was driving? It's like a little diffuse sense of body and some murky perceptions floating through. Maybe some confused abstract idea that we don't really understand. That's kind of the consciousness that... [1:07:59] can fit into this coconut sized biological organ that we think and we think wow we are so conscious but like imagine this Dyson Sphere consciousness or like a mind that maybe spans a galaxy. [1:08:12] I think... [1:08:13] the difference between the sort of awareness that it could have and our awareness might be like bigger than the difference between our awareness and like whatever a flea has or something like that way bigger. [1:08:25] It could be this like... [1:08:27] transition where we develop super intelligent minds that for the first time is really life waking up and becoming truly aware. [1:08:36] and that we are a little bit sort of [1:08:39] overproud in our own specialness when we think that we have achieved something close to the maximum level, that we are the standard by which consciousness should be measured.

[1:08:52] and we're this kind of [1:08:53] feeble, confused, murky. [1:08:56] glimmer that is barely sentient at all. So that's like I think maybe the big [1:09:03] challenge for our era, like giving birth to super intelligence and then hopefully shaping and nurturing it and steering it so that it becomes a positive thing, both for us, ourselves, and also for it. [1:09:19] itself and also for whatever other if there are other super beings somewhere in in the world or in the cosmos that it sort of [1:09:27] is able to get along with us and contribute positively at a cosmic scale. [1:09:34] And that's a very multifaceted challenge, but I think that's kind of... [1:09:39] seems to be what's going on. Yeah, and it's hard for people to think that far ahead. You just think of intelligence being something superior to what we currently experience. But when you're talking about [1:09:53] a computer or a being, a conscious being that is infinitely more powerful than anything we can imagine. [1:10:03] That seems to be what – if everything keeps going in the same general direction and AI increases its power and we figure out new ways to power it. And then – because one of the things that AI needs that's so interesting is it needs enormous amounts of power. And so just these AI centers that they're developing now, they're – like Google is doing one where they're developing their own nuclear power. Do you want some coffee? Hey, sir. Yeah, right here. I saw you reaching for your cup.

[1:10:33] drinking out of paper cups anyway, man. I don't usually do that. I know. I don't either, but occasionally I do. But they're so bad for you. I'm usually quite conscientious with becoming little dictators. Even when I go on the road, I've started taking a little Yeti coffee cup with me so I could buy coffee and just have them put it in that. [1:10:53] The inside of those things is just lined with plastic. You're pouring hot water into plastic, and then the plastic leaches into your coffee. I don't do that normally. [1:11:03] for having a chat with you on the Joragan show, it would be worth, like, I guess, some hundred microplastics. [1:11:12] Well, thank you. [1:11:15] what we're talking about [1:11:17] is inevitable if human beings don't blow ourselves up. If we don't get hit with an asteroid, blow ourselves up, or a super volcano doesn't eradicate civilization, all this stuff is inevitable. It's just how much time does it take? [1:11:31] And how much does it grow exponentially in power? [1:11:35] Because... [1:11:36] We're talking about computers. Then they start bringing up quantum computing and quantum computers' ability to do calculations, it doesn't even make sense. [1:11:46] And so you think, well, this is just one version of that. What is quantum computing going to look like 500 years from now? What is computing power, which is connected to AI? What is that going to look like 500 years from now? It's impossible to even guess. [1:12:02] Well, we can sort of see lower bounds on what's possible, like...

[1:12:07] thinking already of just the designs we can conceive of that we see in principle you could make. Maybe it will take a long, a lot of grind to get there. But at least that establishes like a lower bound of what a technologically mature civilization could do. And then maybe they have additional ideas beyond that. But already that is enough to really unlock. So if you think of a space of possible modes of being... [1:12:32] where like a mode of being is a way of experiencing, living, interacting. I think [1:12:38] You look around... [1:12:39] humanity and all people have existed in the past a lot of different characters right men women like mean people nasty people crazy people but i think all of that diversity of human experience is like a tiny little corner in in in this space of possible modes of being like it's a huge cathedral and we've been kind of basically sitting in in the janitor's closet that's like the the exploration we've done and and the kinds of [1:13:08] modes of being, the kinds of ways of experiencing and relating to each other or thinking and doing stuff that are ultimately possible is just this enormous space that we haven't been able to explore. Because ultimately what we can currently conceive of and imagine and experience and feel is like limited to our biological substrate, the human brain. And just as you're [1:13:32] like earlier human ancestors from a few million years ago that you were talking about before, like they wouldn't really have been able to

[1:13:41] conceive of the modern human condition, [1:13:43] Not just because I didn't think of it, but because like the ape brain can't really, they don't, [1:13:48] like [1:13:49] Even if you try to describe it to them somehow, like what is music, what is humor, what is romantic love, what is science... [1:13:58] what is all of these things, what's literature, like they just doesn't fit into. - Right. - Right. And so presumably, I mean, you could think right now, we've achieved the right brain size for all possible values and interesting ideas. [1:14:12] are accessible to us, that seems kind of impossible. It seems much more likely that just as the chimpanzees are necessarily blind to [1:14:22] like a lot of what can give life. [1:14:24] meaning and value and significance, that probably are we too. Yeah, they don't even have the capacity for it, which is interesting. They can't even consider it. Yeah, we think of them as being so intelligent, and they are in comparison to a lot of other animals. But I think one of the things that they were puzzled by when they started teaching primates sign language was that they never asked questions. [1:14:46] Like, they don't seem to have questions. [1:14:48] about stuff. [1:14:49] So they just... [1:14:51] They exist in an intelligent way. They can figure things out. They can do things. They know when the door is left unlocked. They understand fairness, which is really weird. Like chimps get very upset if they're treated unfairly. Like if one chimp gets something, they get super violent. But they don't have a lot of questions.

[1:15:12] And we're filled with questions. So what are we missing that the next stage of being a human will have that we're not even considering now because it's not a part of our experience? [1:15:24] asking questions. [1:15:26] type of thing that we are not able to imagine. And I'm at many of those types of things. Yeah. And the possibilities are literally endless. [1:15:34] We just don't conceive of them because we're trapped in this territorial primate mindset. [1:15:41] Yeah, yeah. I think that's likely. [1:15:45] Yeah, I think so too. I think so too. I think the possibilities are endless. And I think it's going to be really, really weird if that happens inside of our lifetime. [1:15:56] Because you and I were close to the same age, so we grew up. [1:16:00] With answering machines, you probably remember the first cell phones. You probably got online and probably – when did you get online? Like the early – Ninety-six. Yeah, there you go. Yeah, see. I think for me it was 94. So we got to see all this – [1:16:18] happen if we get to see the new version of humans in our lifetime that would be like literally bonkers [1:16:27] What an amazing time to be alive if you really think about it. We're so fortunate because if you grew up between like – [1:16:33] 1700 and 1800, how much did shit change? 1600 or 1700, would you make a better boat? Like what's different? Not a lot of different, man. Everybody kind of is the same for 100 years. And inside of our lifetime from the late 1960s for me –

[1:16:51] to the early 70s for you, like in the amount of time? [1:16:55] that we've been alive, things have radically changed. [1:16:58] Like really, really radically to the point where it's probably the biggest shift in human ingenuity and innovation that the world has ever seen. And we're just in the middle of it. And we might be in the middle of the next one, which like literally allows us to see what the world looks like a thousand years from now because you're going to be alive. I mean that's why it is now this full-time job just to monitor the situation. How do you do that? [1:17:28] Well... [1:17:29] you don't really but if you try to sort of [1:17:34] So I think the opportunity – so it used to be – [1:17:37] Um, [1:17:38] Like, at least in my sphere, I think if you're doing philosophy or something like it, um, [1:17:44] most people would think you have a kind of unlimited time horizon. People have been working on philosophical questions for thousands of years, and [1:17:53] there doesn't seem to be any huge [1:17:55] urgency if they have been unsold for thousands of years maybe if it takes another 500 or so but i always thought of philosophy as having a deadline [1:18:05] Um... [1:18:06] meaning that at some point, [1:18:08] we would develop smarter than human forms of intelligence, presumably AI, that could then do the philosophy much better than us. And so that was a limited period of time during which any advances could be,

[1:18:22] like I could contribute to would be meaningful and that it would make sense on focusing on that subset of philosophical questions. [1:18:29] general questions that we really need the answer to now as opposed to like, you know, [1:18:34] 10, 20 years in the future when somebody else can do them better, like the machine minds. So that's kind of been a lens through which I have selected the things to work on. And now, of course, that deadline is moving closer. So there's less time remaining. And so my focus is... [1:18:52] increasingly drifting towards questions where it might be relevant to have the answer to now rather than a year or two from now. You're almost like a cultural navigator, like a guy with a sextant at the helm of a ship. [1:19:06] Thank you. [1:19:06] looking at the constellations you go I think we're in the right direction yeah but now we're sort of moving maybe into close to harbor and you need to like yeah look pay more attention to exactly how deep the rocks are the rocks are and yeah scan around you what when you think about a timeline for radical change what in your mind is [1:19:32] What do you think that looks like? Well, I mean, I take short timelines seriously with AI. I mean, for what we know, it could be like a year or two or three or four and probably a bit longer. [1:19:47] But we're no longer at a point where we can be confident that we won't have superintelligence in just a few years. Like it could happen.

[1:19:56] What is that? When you say that for the uninitiated, what do you mean by superintelligence? Well... [1:20:02] I guess... [1:20:04] We first have AGI, Artificial General Intelligence, AI that can do [1:20:11] all the stuff that we can do. [1:20:13] And then super intelligence would just be that, but can do it way better than any human can do. [1:20:20] So all technical, intellectual, [1:20:23] common sense tasks and then [1:20:25] and they would have robotics as well, looking at all the physical stuff, not much later. [1:20:32] - [1:20:32] and um [1:20:34] So this, yeah, so the timeline remains uncertain, but... [1:20:38] I think it's not impossible that this could happen very soon. And then once you have super intelligence, then I think from there on, it might be like a sort of sprint to... [1:20:48] something approximating technological maturity. Because what you have super intelligence that then designs even more smart AIs, right? Using its kind of super intelligent AI research capability and designing better chips and all of that. [1:21:01] So you might then have this like intelligence explosion [1:21:06] where you go from something slightly greater than [1:21:09] human level [1:21:10] to some radical super intelligence that can then sort of invent whatever the remaining technologies are. Maybe there needs to be some trial and error and experiments in the physical world that slows things down a bit. But some smallish number, like a single digit number of years from super intelligence, I think you might have something that unlocks all these sci-fi level capabilities that we've talked about.

[1:21:31] Um... [1:21:33] At least that seems relatively plausible to me. [1:21:37] Um... [1:21:39] It seems inevitable. [1:21:40] And my question is, how does it? [1:21:43] How does it announce itself? Does it send a mass text message to the whole world? Everybody's phone just starts like, you know, when you have those amber alerts and your phone starts vibrating or when there's some sort of a storm warning. [1:21:56] All of a sudden your phone goes off and then it alerts you to the fact that it's taken over. Well, so this means so here we don't really know. It's like this is very confusing and we've never done this before. And so it's like very hard to figure out how this is going to unfold. [1:22:13] depends a little bit on what we do and the extent to which different actors [1:22:18] get their stuff together. Yeah. Um, [1:22:21] But... [1:22:22] Um... [1:22:25] Like one, I guess... [1:22:27] One possible type of scenario is where things are just accelerating. There are more and more of these advances, model releases. Increasingly, there is automation of the research process [1:22:39] process in AI labs. [1:22:41] like already you have coding assistants, right, that are really useful to people in AI labs. They're using them to write a lot of their software. Like right now you still need humans for sort of the research taste and judgment and, [1:22:53] Sometimes things get stuck and you need a human to kind of redirect them. But more and more of that. [1:23:01] may get automated.

[1:23:02] And then [1:23:04] you have kind of... [1:23:08] new iterations of models being trained [1:23:11] at the faster and faster clip, they can do more and more stuff. [1:23:15] Thank you. [1:23:16] they start to... [1:23:19] Automate [1:23:21] increasing chunks of the economy. [1:23:23] So right now a lot of coding is automated, but like other areas as well, maybe they become drop-in virtual workers that can do everything like a human could do with a remote connection. [1:23:36] initially and then like you have people working on robotics so that that would start to kicking and eventually just more and more of the action is is run by these ai systems one or more um [1:23:47] And they're kind of [1:23:49] doing it at their timescale, which is speeding up. [1:23:51] then from that point it would depend a lot on whether we have successfully aligned these systems and [1:24:01] so that they actually do what [1:24:03] the people created them intended or whether they have somehow gone off the rails. [1:24:08] There's also the fear that America doesn't come up with it first. [1:24:16] This is out of fear over here. Yeah, that's mostly a fear in America. In America, yeah. Like in China, for example. Yeah, America comes up with it first, yeah. I mean, it's really just America and China that are at the forefront of this race, right? Yeah. Russia a little bit, but not on the same level. Not Russia. No? Not Russia? Who else is it? Did you see this last week? Ford hiring 350 engineers after AI failed shows...

[1:24:40] Failed shows human value in AI era. What does that mean? I tried to find the only one that didn't. I just did it. [1:24:46] They rehired a bunch of engineers they fired after their AI wasn't matching the quality they needed. Oh, and this is Ford. That's interesting. They probably jumped the gun. [1:24:58] You know, they probably fired him too quickly. [1:25:02] And now these people that they rehire, these people are going to like, oh, how much time we have left? Six months before AI figures out how to do this just as well? [1:25:11] it's also possible that [1:25:13] things time-wise could extend if... [1:25:17] but [1:25:18] One way that could happen is the progress we've seen in the last years, [1:25:23] 10 years. [1:25:25] which has been remarkable, right? It's to a substantial extent driven by the advances, the increase in compute that is being used to train and run these systems. [1:25:35] It's like it used to be... [1:25:36] that's [1:25:37] You had a cutting-edge AI system. If you were an academic, like running on your PC in your office, that was... [1:25:44] kind of the amount of computing power that was applied to like doing AI stuff. And now, of course, you have these kind of tens of billions of dollar... [1:25:52] data centers, like hundreds of megawatts. Right. Like just massive funding. So the chips have gotten better, but also just the amount of funding, like you're just building many more of these chips. And so as you apply more and more compute, like performance improves significantly. [1:26:06] And that's like has been a big driver. [1:26:09] Now, at some point...

[1:26:10] you might not be able to keep increasing the amount of money you spend on it. Because like, [1:26:17] You can go from like a thousand dollar PC to like a million dollars quite easily. And you can go from a million dollar to a billion dollars. [1:26:25] And now maybe you're spending on the order of a trillion dollars across the world to build data centers per year. But you can't really... [1:26:34] do like three orders of magnitude very easily there. There's like not a thousand trillion dollars to spend on it. So at some point that expenditure has to kind of slow down. So if we haven't achieved super intelligence by then, then maybe that would mean progress. [1:26:47] gets slower if the main driver is the scale-up of compute. Now, it is also true that some of the progress is driven by algorithmic advances, like just kind of clever algorithms. So that might continue. But if one driver stalls out, then... [1:27:02] that could result in faster progress. And then of course, there is a possibility that, um, [1:27:08] The people who want to pass or regulate AI gain enough traction to kind of get regulatory inhibitions. [1:27:16] How would they do that though if there really is some sort of a Manhattan Project style race between the United States and China? And what other countries are developing AI right now that are close? Well, I mean those are the – Those are the two big ones? The big ones. Yeah. [1:27:29] Is it possible that someone could sneak up on us and – [1:27:33] Develop super-intelligent AI first? Yeah. I mean, it's possible if there were some big algorithmic breakthrough, for example, that made it a lot more efficient to run a similar level of capability with less sort of AI data center infrastructure. And many other countries are also trying. It's just that they are not...

[1:27:52] as advanced and and and out of US and China I'd say like US currently has the edge and [1:27:58] What would happen if China got there first? [1:28:01] Wow, I mean... [1:28:03] Part would depend on whether they had successfully aligned [1:28:06] for their AI. [1:28:08] If it's unaligned, then I guess the same thing happens as if USAI is unaligned. That is, the future gets shaped according to whatever... [1:28:16] values this AI had ended up with. If the alignment problem is solved, then it might make a difference because then... [1:28:24] the values would then depend on sort of the people who owned and controlled or governed it. [1:28:32] would ask it to pursue. So then in that scenario maybe it makes some difference who [1:28:39] initiated. [1:28:41] Um... [1:28:42] So right now the big players are – there's Google and there's OpenAI. There's all these different companies. When you say aligned, do they have to all be kind of on the same page or like when aligned – we're not going to be aligned if there's a bunch of corporations that are competing to come up with this first. So they have to be aligned in terms of the way they're programmed, that they're valuing human life and that they're valuing society. Like what do you mean by aligned? [1:29:12] talents of [1:29:13] if you're building an AI system, [1:29:16] and you have certain things you want it to do and certain things you don't want it to do. Right. Are you technically able to get the AI to behave that way that you as the person... And who gets to decide how... Well, that's a separate question, which is equally important maybe, but different, because that's not the technical... Like you can't just go to the whiteboard and write down some formula. Now you have... That's like a political question ultimately, a question of governance. Yes. So there you need...

[1:29:41] you know [1:29:42] political organization appeals to the best in human nature, dialogues, like checks and balances, whatever stuff that might work in the political arena. Um, [1:29:53] to hope that the governance of this, that the values to which it is aligned are sort of benign values that hopefully incorporates [1:30:01] a wide set of stakeholders. Right. But that's a little isn't that a little naive? Because whenever there's any sort of a situation where something has massive amounts of power above others, that one of the first things I think of is what's the most money you can generate? [1:30:15] doing it [1:30:17] And what is the best way to generate the most amount of money? Like that's – [1:30:21] They're going to think that way. They're not going to think like, what's the best for the human race? Like no one ever thinks that way. They think what is the best way in terms of like destroying the competition? What is the best way in terms of extracting the most resources out of this? [1:30:34] making it work for me. That's what people think of immediately. And what's the best way to stop other people from competing with us? Right. Well, so that would be, I think, a lot of – [1:30:47] pressures and strains on whatever governance mechanism. [1:30:52] exists at a time when superintelligence [1:30:55] is developed. [1:30:56] Right. So is this something that the technology is so far ahead, the potential for it to be so far ahead of our understanding of what it's going to be able to do that like making laws for that now, it's going to be very difficult to even explain to people. We need these laws and we need these guardrails in place now because here's what could happen. And that conversation is not really happening right now. It is happening. But is it happening politically?

[1:31:26] you [1:31:27] It's starting to happen, yeah. Yeah. It's been happening to an increasing extent politically, and there have been various actions. Okay. Like what actions? Well, so a few years ago, for example, that was the whole export... [1:31:41] regime imposed by the US on chips, the most advanced chips, where like China was cut out from being able to access the most advanced NVIDIA chips and so forth. And [1:31:53] And that's kind of been modified, but that was motivated to a significant extent by trying to preserve information. [1:32:00] AI [1:32:01] edge that the US has. Right, but that's like internationally. I mean, in terms of like being... [1:32:07] able to stop corporations from having the kind of power that this could provide them. So then there have been various efforts that was like – [1:32:16] proposition in California where there were various things. [1:32:21] More recently, you saw the whole thing. I don't know if you followed it with the Mythos and Fable 5, the anthropic models that – [1:32:31] So Mythos has not been released. This is the most powerful model because... [1:32:36] It seems to have... [1:32:37] significant cyber offense capabilities. It can easily detect vulnerabilities in software. And so [1:32:45] Anthropic figured that rather than immediately making it available to everybody, maybe it would be better to first try to make it available to providers of – [1:32:55] critical software infrastructure, like big banks, and I'd like that they can patch up their systems.

[1:33:03] Um, [1:33:04] Um... [1:33:06] And then... [1:33:08] Fable 5 was a kind of restricted version of the mythos model. [1:33:12] like the same underlying model, but with extra safeguards. It basically refused anything that remotely seemed like, you know, cyber... [1:33:21] Hacking, programming, biological stuff, because maybe that would be bio. So like it sort of drew a wide circle around anything that even remotely looked possibly dangerous. And it just refuses that. So that was released. But then after like a week or so, the administration imposed an. [1:33:43] a kind of export restriction that prevented any non-US citizen [1:33:48] from using it. And that meant Anthropic had to cut it off for everybody because they didn't have a way in real time to verify who is a US citizen or not. - Oh wow. - So then it was like unavailable for several weeks and intense negotiations behind the scenes and working to try to figure out. Because allegedly it was possible to jailbreak it so that it sometimes gave some little assistance with some cyber, like finding vulnerabilities in code. [1:34:16] And now recently it just became available again because they had reached some understanding with [1:34:21] with the government, where it was deemed sufficiently safeguarded to be released. [1:34:28] And now there's like efforts underway to try to work out the framework because like in the future you wouldn't want to do this on an ad hoc basis like somebody just decides this particular model for some unspecified reason like you want to have clear standards.

[1:34:41] right? That applies to everybody, every company, [1:34:43] So there's now some industry-wide effort to try to work with the government. Anyway, so there's like a lot of this stuff happening. I expect much more. [1:34:52] of this going forward. It just has recently become like a serious issue. People until recently were kind of ignoring the whole AI thing for the most part. [1:35:01] and [1:35:04] So, [1:35:04] Then there is a second wave coming, I think, once so far we haven't really seen any big impact on the labor market. [1:35:11] right, from AI. But once that starts to hit and you get maybe [1:35:16] you know [1:35:17] high levels of unemployment amongst white-collar workers. [1:35:21] Imagine if you have millions and millions of people with... [1:35:24] They have their university diplomas, right? They feel a sense of superiority and entitlement. They've gone through the whole process. They got their degree, and now they expect a well-paying job. [1:35:35] and then there is no job for them. [1:35:37] They've got nothing to do all day long. [1:35:40] but complain about AI. [1:35:43] So you've got to have all these well-educated people who feel resentful and are going to say every possible bad thing about AI that could be said all day long and mobilized. There's a very powerful political constituency that will emerge from that. That's not even yet happened, but that will kind of add to all of these other issues. [1:36:00] grievances that people point to with AI. So I think there's going to be kind of significant political pressures for AI. [1:36:09] doing something about AI. Isn't the key is getting ahead of it, though? So, like...

[1:36:16] How can people find – how can we see the vulnerabilities in advance and recognize like when this is going to – like if there is going to be a tipping point and a bunch of white-collar workers are going to be out of work and there's going to get to a point where we realize like this is coming. This is like three weeks from now. What do we do? Like what do they do? Yeah, so that's not clear to me at least. I mean a lot of people think – [1:36:40] They have different views about what should be done, of course, but it's like very hard to have. [1:36:47] It's such a complex strategy. I mean, I've been thinking about this for like... [1:36:51] three decades maybe, and I still feel extremely unsure even which direction is kind of up and which is down. [1:37:00] I don't think it would be possible right now to sort of figure out a detailed, you know, perfect regulatory scheme or system of laws. [1:37:08] because there's so many unknowns. I think we need to sort of watch this closely as it happens and be ready to react quickly to... [1:37:16] issues.com up. [1:37:17] Um... [1:37:18] and hope that relevant people are like [1:37:21] highly competent and well motivated and are trying their best. And then on the margin, maybe we can do things that kind of are constructive and increases the chances that it will be for the good. [1:37:34] um [1:37:36] If and when I have some clear insight as to some overall big directional push, I will... [1:37:43] let you know.

[1:37:46] Thank you. [1:37:46] But even basic [1:37:48] questions like for example [1:37:50] Do we want more government involvement or less government involvement? [1:37:53] Do we want... [1:37:54] faster progress in AI or slower progress in AI? [1:38:00] Um... [1:38:02] I don't know. Uh, [1:38:04] It's not completely obvious to me. Well, I think it's eventually going to get to wherever it's going to get. [1:38:11] Either way. [1:38:12] Having it slower... [1:38:14] I don't know if that's really going to help us. I think almost like we have to crash and then we have to figure out how to – [1:38:20] rebuild and pick up the pieces. I don't think we're going to be intelligent or have enough foresight to recognize where all these flaws and where all these problems are going to ultimately be. I think they're just going to have to happen and then people are going to have to adjust. [1:38:36] That's what I think. Yeah. So I'm not... [1:38:40] advocating an AI pass by contrast to a lot of [1:38:43] some of my colleagues and friends and stuff who are like, I could see some scenario in which [1:38:50] it would be helpful at some point [1:38:52] to have a temporary slowdown of, you know, a few months maybe, or a year or a year. Like if you imagine, you know, there are different companies, countries maybe racing to get there first. [1:39:02] And then eventually somebody figures out they basically have the system in place. They just need to... [1:39:07] like amp up like run it for longer they can sort of see that it will become super intolerant. They hope it's aligned. They hope it's aligned. [1:39:14] It might be very helpful in that situation to have a few extra months just to sort of double check all your safeguards and rather than immediately cranking all the knobs up to 11, like maybe sort of do it a little bit incrementally, watch what happens, study it, then crank it up a little bit more.

[1:39:32] I just think you might gain some extra safety if you have a few extra months there, like the pressures on these people in the lab. That makes sense. If it's all going to happen over a week, like it's just going to be, this is the first time we ever do this hugely complex thing. It has to be right. We never get a second chance. Just being able to, like even just being able to sleep properly for like between your work sessions and like having... [1:39:53] you know, a weekend to mull things over. Like just that kind of [1:39:57] humans don't really operate on... [1:40:00] well in these super short timelines. So I think [1:40:05] short [1:40:06] If you could be sure that the pause would in fact be... [1:40:09] limited and short and then it would be lifted i think that could be quite useful to have at if it could also be timed to be at the right moment and well implemented but wouldn't the problem be espionage first of all that's one one issue with it yeah if they realize if somebody realized hey they're about three days away from this [1:40:28] And then China... [1:40:30] bribes a bunch of people and you know people take off and [1:40:34] move to... Yeah, there are various downsides to pulsing. So one is that... [1:40:42] like their competitors who just even without the spionage or catching up. Another is [1:40:48] It's not as if the world is... [1:40:50] safe. [1:40:52] Without AI. Right. Good point. There are other – like the natural hazards, I think they're very small, like supervolcanoes and stuff like that on the relevant timescales. Until they're not. Well, I mean we've survived for hundreds of thousands of years, right? We're almost wiped out entirely 70,000 years ago. Yeah, well, that's a long time ago. Is it though? Well. Not if it happens tomorrow, it's not. Well, like if something hasn't done us in –

[1:41:17] in that long a time, probably it's not going to do a thing within the next 10 years. Boy, you're more optimistic than me. Well, it's just based on the actual... [1:41:25] empirics of these natural catastrophes. By contrast, I would say there are other hazards that are not low. I mean, we have rapid advances in synthetic biology. [1:41:36] some of them driven or enabled maybe even by [1:41:39] already... [1:41:40] um um ai progress that has already taken place like that can start to assist with this and and more just happening independently so you get increasing risks from synthetic biology we still have the nuclear arsenals right kind of existing um [1:41:57] We, I think, have gotten a little bit complacent about the risks of nuclear war. [1:42:01] Um... [1:42:02] um [1:42:04] And... [1:42:06] And various other things as well. So I think there's like a background level of existential risk that humanity faces in the absence of super intolerance that probably is growing as well. So you don't want to wait so long that you don't even get the chance to roll the dice with AI because you destroyed yourself. [1:42:23] before I even got it. We can't be sad. [1:42:27] And then... [1:42:29] So [1:42:30] Then also, I mean, I think there is like some risk if you set up a pause infrastructure that what's initially meant to be a temporary infrastructure. [1:42:38] becomes permanent. Like they say, there's nothing more permanent than a temporary government program. So if you set up the infrastructure to actually control this, ideally at a global scale, right? It's meant to be for six months. Like what happens after six months? Nobody can still prove that the AI will be safe. So you have all these people now

[1:42:56] whose job it is to regulate it. And like maybe they, so it could become kind of frozen in and become permanent. [1:43:04] And then also... [1:43:07] So [1:43:07] Mm-hmm. [1:43:08] the delay [1:43:09] benefits. [1:43:11] So if you think what like 65 million people die every year? [1:43:18] That's... [1:43:19] That's a lot of human lives. [1:43:22] That's like... [1:43:23] One 9/11 every 25 minutes just kind of boom like that's a way to look at it. Um, I [1:43:30] And so there is a certain urgency. If there is something that somehow could [1:43:34] have a chance to [1:43:35] to... [1:43:36] to fix [1:43:38] this, like all the suffering that is happening in the world, like aside from the people dying, like all the people who are bereaved, right, who lost their loved one and then all the disease that led up to that dying. [1:43:49] Right, and then all the non- [1:43:51] disease related, all the horrible poverty and like the suicidal depression and the animals in the animal factories that are like spending. There's just like this... [1:44:01] I think moral urgency that if there is a hope that [1:44:05] Getting... [1:44:06] AI right could fix this then you don't want to wait unnecessarily long because every day is just this massive horror. Right. Um, [1:44:13] So I think there are many reasons for why you wouldn't want unnecessary or excessive. [1:44:18] delays in developing AI. But there is a trade-off because if we can make more progress on AI safety and alignment and get our act together a little bit before we take the plunge, that's also worth quite a lot.

[1:44:31] I [1:44:32] It's... [1:44:34] It's certainly exciting. [1:44:36] Because the possibilities... [1:44:39] are right in front of us and they're kind of endless. [1:44:41] And it seems like they're right in front of us. It seems like, in my mind, it seems like we're like 24 months away from something really insane. [1:44:48] could be, could be, could be 48 months it could be, I mean we don't [1:44:53] Yeah, right. It could be. Yeah, who knows? But there's something happening really quickly. You know, when I talked to Elon about Grok, he said it's it shocks us like every couple of weeks. [1:45:05] We were like, how is it doing this? How is it so fast? How is it advancing so quickly? [1:45:10] And one day... [1:45:13] Kat's going to be out of the bag and there's not going to be any way to stop it. [1:45:16] Like if the power went out right now, if there was some sort of a massive solar flare that killed our power grids and all computer hard drives got fried and we had a restart from now, like – [1:45:30] we would have to rebuild society, right? But if we get to the point where this thing – [1:45:36] understands what would cause that, how to prevent that, how to make sure that never happens, much better power supply, much better allocation of resources, much better batteries, much better redundant data systems where you never have to worry about – [1:45:54] Hard drives crashing and you never have to worry about any of these problems. Like any information you have now will be secure. And then understanding of natural disasters will actually drill into this volcano and it never goes off. Like there's going to be a few things that they're going to figure out through AI that's going to prevent a lot of the things that are probably wiped out. Enormous swaths of people. I mean imagine if we could just actually see asteroids coming, all of them, and know how to divert them instantaneously. Yeah.

[1:46:21] We have a bunch of ideas right now on how to do it, right? Like coding them with something that changes the aerodynamics of them. I think that one would be relatively easy. I mean even we could figure out to do that, I think. And there's also detonating them. There's a bunch of different crazy ideas. But imagine if AI is like, actually, you could just do this. And then you put a shield over the earth and you never have to worry about it ever. The shield is powered by the sun, so you have ultimate power. [1:46:51] being hit by a Manhattan-sized asteroid. I think the trickier ones might be ones that are more internal to civilization. [1:46:59] If you have... [1:47:02] some [1:47:03] process that's either like a worldwide process with different humans and corporations and governments or like an internal process in the AI like on the one hand you want to be able to continue to develop new ideas, new ways of doing things. So you need to experiment and try new things. But then there's also the risk of [1:47:23] if you just keep [1:47:24] trying new ideas like that, eventually you get stuck on some idea that actually proves really harmful or dangerous to you. [1:47:31] Um... [1:47:32] um, [1:47:33] and that you then sort of derail internally through your own memetic or internal evolutionary development. So... [1:47:42] how to sort of grow up [1:47:45] safely. [1:47:46] in this world where you have unlocked all kinds of new technological capabilities, even if you're easily able to protect yourself against external threats like volcanoes and asteroids and stuff. Right. There might still be processes arising from within this global civilization. Right.

[1:48:03] just as we might invent new technologies that are dangerous, we might invent new like drugs that are super addictive, once you invent them, [1:48:11] you try them and then you're addicted, you never want to [1:48:13] Yeah. There could be other like some some crazy ideology that once it takes hold. [1:48:19] I'm glad you brought that up because that's one of the major fears that a lot of people have is that human beings will develop ideologies based around AI, that there will be like an AI god that someone develops. [1:48:32] Yeah, and I mean, I think that's in fact, I would add that to the list of I've mentioned nuclear and bio. [1:48:40] terrorist risk as like background existential risks. Like another is like some form of insanity, like collective insanity. I feel our civilizational sanity already is kind of a little bit precarious. I think we're just kind of maybe barely holding ourselves together. And it's amazing how well the world functions. [1:49:03] despite how crazy people are and how much they loathe their opponents and so forth. And we still manage to somehow get it to work. There's also manipulation of the zeitgeist that's clearly being done by bots. So you've got a bunch of people online that are having arguments on Twitter, and they're not even talking to people. There's a bunch of ideas that are being pushed out on Twitter and a lot of these social media platforms. [1:49:28] It's not even human beings tweeting about it. It's not. There's algorithms. There's AI. There's people that are hired to do it. There's people that are working for certain organizations that are hired to muddy up the waters, gaslight people, create problems, have people argue with each other.

[1:49:48] If AI recognizes how easy it is to manipulate people, do whatever it wants, that's a fear as well. [1:49:57] change the basic parameters of sort of the [1:50:01] social... [1:50:04] cultural, political discourse. [1:50:06] New Dynamics will emerge [1:50:10] And we don't have... [1:50:12] the kind of social science that is able to predict what happens if you change some of these knobs. So we've seen in the past, like you invent, let's go all the way back, like somebody invents writing. [1:50:21] Okay. [1:50:23] turns out to have had a huge effect [1:50:26] not just on people writing literature and stuff, but on political systems. So you could now have states that could keep tax records, right? [1:50:33] Thank you. [1:50:34] So you can have larger political units with writing, and then they can hire standing armies, [1:50:39] and now you have like large-scale war, you have social stratification, you could have like the ruler of a large area, could have enormous wealth and you could have [1:50:49] a soldier cast that like protects against internal and external. So like just the way that, [1:50:54] human societies are politically organized changes as a result of this change in the rules of [1:51:00] communication when you can have written texts, right? And then you have the printing press again, we're like a hundred years of religious wars in Europe like reformation and all of this stuff is possibly and then like I [1:51:13] forms of democracy later on, also coming out of this, and the scientific revolution.

[1:51:19] Then you have like mass media in the last century with like radio and stuff. And you have kind of demagogues that take advantage of being able to simultaneously talk to millions of people. [1:51:29] because that was never possible before, right? - Right. - You could have some charismatic kind of guy who's like rallies up the whole nation and then, [1:51:36] new ideologies become [1:51:38] memetically fit in that situation that might never have if it were people writing kind of [1:51:44] letters to the editor. Like it's kind of a different type of idea that works there than if you're giving a kind of stump speech that goes out to millions simultaneously. And now with social media, of course, we have another one of these and you do see that starting to change culture. [1:51:57] in different ways. [1:51:59] And now with AI being the next wave of this, that will also change. You have like bots. You have new ways of finding information. You have maybe AIs that can themselves be super persuaders. That will also change. [1:52:11] presumably in some unknowable way, the way that... [1:52:15] social discourse pans out. For any one of these, I guess it could turn out to be a lot better. We could become more informed having AI advisors. I think that's fairly likely, Sinoja, but it's also possible somehow the dynamics shape out in a different way and we kind of go collectively insane in some way. Back to the whitewater raft. White to backwater raft, yes. And it all becomes kind of totalitarian or we sort of fragment into... [1:52:41] like political warring tribes, or we become kind of completely unhinged. Every one of us becomes convinced of their own little nutty theory that they then, like their AI or feed just serves them more material to kind of fuel their conviction. There are different ways in which this could...

[1:52:57] go badly. But don't you think even if that happens, ultimately the progress of AI won't stop? [1:53:06] And so... [1:53:08] Again, I keep going back to this thing, but I think this is really what it is, is we have to change, like what it means to be a person. All those things are only problems if people stay what we are now, which is territorial primates with desire to – [1:53:24] possess material goods for some strange reason, even though we're a finite life form. If all that changes, if we change what it means to be a person, which seems inevitable, [1:53:35] then it won't matter. If we all... If we could figure out a way to literally engineer out [1:53:46] all of the issues that humans have with greed and violence and all the different things that trouble society. [1:53:52] If the desire for that is no longer a part of being a person. [1:53:57] which is... [1:53:58] That's doable. That seems like if we're going to continue to evolve past, you know, Australopithecus to Homo sapiens 2026, if we continue – [1:54:08] The same amount of time in the future probably won't be like that anymore. The best versions of people aren't people that want to steal your house and steal your land and shoot you and take your resources. The best people are the people that contribute and they're interesting and they make you excited to be around them and you like it. If that keeps going on and if that is aided by technology, if we recognize that there's actual patterns of human thinking and behavior that can be changed –

[1:54:37] and that if we all agree to... [1:54:41] subscribe to this algorithm, to connect to this computer program, connect to this external device, or maybe not even external, maybe it's internal, that allows us to communicate with each other telepathically. All that changes, and then it doesn't matter. [1:54:59] It doesn't matter if there's any guardrails for AI or not because we're not the same thing anymore. All the problem – the worry that we have about AI, the worry we have about power, manipulation and the ability to influence people. [1:55:12] All that kind of goes away if people don't behave the way they behave currently. [1:55:19] I guess here's one... [1:55:20] difference with your like [1:55:22] white water rafting metaphor. So in that metaphor, it's kind of [1:55:28] We need to hang on. Like there's a risk we could smash ourselves on the rock. But if we don't do that, then there's kind of one way we end up right downstream. We get up in a nice lake. Yeah. But I think in reality, what might also be the case that in addition to trying to not smash ourselves on the rock. [1:55:44] that might be places where the choices we make affect the ultimate destination. [1:55:50] Right. [1:55:52] Go left or go right. Yeah. Or which stream, but maybe, maybe a more, yeah. And where like, if you stare in one direction, you might end up in one place ultimately in some sort of strange post-human world. And maybe it's really beautiful and people have the chance to grow into their true self and we love each other and are creative and take initiatives and go a different way. Maybe you have a paperclip maximizing AI or maybe that just everything is...

[1:56:18] So going a third way, maybe you have like a totalitarian system with like a small elite on top and everybody else. Yeah. Or just. Sci-fi movie. Yeah. And there are many more possibilities. And maybe some that kind of we would. [1:56:34] think would look kind of good if we chose now, but then... [1:56:37] In reality, they have some hidden flaw that we didn't think of. So if we chose those, it would sort of be a mistake. And then maybe others that don't look that appealing to us now, if you just presented them in a brochure to you, like, but then actually, if you thought hard about it, maybe you would realize that actually would be a really nice place to live. [1:56:54] Because like in the current world, there are places that are nice to visit, that are interesting, but then those are not necessarily the same places we would want to live and raise a kid, right? So there are like novels and movies that are really fun to watch. [1:57:08] But you wouldn't want to live in those worlds. So there's a difference between what makes for the good, interesting story and the place where you actually want to spend the rest of your life. And so I think that would be... [1:57:18] Possibly a lot of opportunities to make foolish choices or unwise choices or conflicts that sort of thwarts. [1:57:26] the process and makes us end up not just that we go extinct before we reach them, but that the deer that we reach might depend on the level of wisdom and kindness that we have during the process. And there's always unintended consequences with every solution that we try to find for any problem that we have. [1:57:42] There's always some new thing that comes up. We're like, oh, we didn't see that coming. [1:57:46] I mean, certainly whenever we're dealing with nature, like whenever they've brought in invasive species to handle other invasive species, it's always a giant disaster.

[1:57:57] You know? [1:57:58] Yeah. Always. There's always unintended consequences like, oh, now you have 100 million frogs. Who's telling us about that? Was it Ali Sadiq that was telling us about the frogs? [1:58:09] Who... [1:58:09] So... [1:58:11] about they brought in, what did they bring in? [1:58:15] I forget what the invasive species was, where they brought in this one species to kill another species, and the problem, they didn't realize that that species had big control. It was like coyotes or – I forget what it was. Coyotes. [1:58:27] It was Guam. What did they get rid of? [1:58:31] whatever it was, they got rid of this thing that had been killing the frogs. So then they had... [1:58:39] Toads, right? Was it toads? [1:58:41] So they had millions of these things that were like all over the highway. So you drive on them, just squash them everywhere you go because there was nothing controlling the population of these things anymore. [1:58:50] It's like unintended consequences of – because we're very short-sighted. And even in our ability to be – [1:59:00] contemplative about what the possibility of the future is. When you're dealing with such an open-ended thing like AI, super general intelligence that can maybe make better versions of itself. Who the hell knows what that means? Are we making a god? Because it seems like if it keeps going and that – if it makes better versions of itself, it's ultimately going to get to the point where it could do anything. [1:59:21] which is exactly what a god is. It can make universes. [1:59:25] What if the ultimate end of this is a big bang button that some scientist invents?

[1:59:32] Yeah, so we are a little bit like, yeah, it's, I don't know, the different... [1:59:38] analogies one might reach for, but I mean, I guess like, [1:59:41] like say you're on a plane and then like [1:59:44] the pilot has passed out or had a heart attack or something. Now it's like we passengers... [1:59:49] We somehow have to, I don't know, figure out how to run the controls. Yeah. We're not really – I mean, you have to still try to make your best, right? But then add to that that all the passengers are fighting amongst each other. Each one thinks they are the guy who should control the – like they're all convinced that they are the superior person to try to land the plane. So in addition to trying to figure out how the controls work, they are like actually having a big fist fight in the cockpit as well. [2:00:19] And there's like the kind of monk tribe that is the current state of the world here. And we're trying to shepherd like this ship of humanity into utopia. It's an interesting... It's Dr. Strangelove on steroids. But that's it. We might only have to like get it so far. At some point, we should be able to hand over like... [2:00:47] Um... [2:00:48] the reins. Once you have a sufficiently good AI, maybe get assistance from that point on. I think it's the first thing that it's going to take over is government. It's going to realize how [2:01:00] unbelievably inefficient the government is at doing almost everything and how much of the money that gets allocated is fraud and waste. If you allow AI to sort through that and develop much more efficient pathways to –

[2:01:17] controlling and so many resources. Some people would not want to see that. Exactly. That's going to be a problem. But if we get to the point where there's... [2:01:26] some sort of some sort of a hive mind possibility. Some sort of a [2:01:33] I mean one of the things that Elon said that I thought was really fascinating, he said, you're going to be able to talk without words. [2:01:39] Well, if we're able to talk without words, like... [2:01:42] Does that eventually get to the point where we could read minds? Is – [2:01:47] is thought and is communication no longer verbal. It's no longer sounds. So right now we associate sounds that I'm making where you know what words I'm using, what I'm referencing, and we get a certain understanding of what each other is trying to say. But what if that's just clunky and that's silly? And what if instead of a scroll that you leave in a cave somewhere, now you have a movie [2:02:17] much more powerful and that this is what [2:02:20] Human communication becomes. It doesn't. And maybe this is why the grays don't have mouths. We move away from sounds. Because right now what we figured out is like sort of like duct tape. We were communicating. We kind of patched it up. We figured out something. We're just going to use noises we make. Well, we have different noises here than the people that live on this island. They have totally different noises. I don't know what the fuck they're saying. And then you have people on the other side of the world. Totally different noises. So Tower of Babel type situation.

[2:02:50] where we really can't communicate with each other unless we have translators. But if we get to the point where that's not how humans communicate, we communicate purely through thought and intention and understanding and that [2:03:04] It's no longer based on language. It's no longer based on this is a transistor. This is a coffee mug. Instead, it's a complete understanding of each individual thing that we're discussing. Everything, you know what it is. You understand what it is without it having to have a noise associated with it. Yeah, I mean, I guess the cyber security implications are significant if you are giving direct access to other people to transmit signals to your brain. [2:03:33] in a high-bandwidth way that is not just words kind of that... [2:03:38] it's almost like you heard them even though there's no sound in your ear, but if it's like actually directly kind of interfacing in a high bandwidth way with your neural network. Also, encryptions out the window. [2:03:50] If we no longer have encryption, if we get to the point where – [2:03:53] Why is that out of the window? Because if computing gets to the point where the bottleneck is – like think about money, right? What is money right now? Money is all ones and zeros somewhere essentially, right? It's all bank accounts. It's like we're not on a gold standard anymore. So what if that bottleneck – [2:04:12] It's an information bottleneck. Like someone is preventing you from going into these places and getting these ones and zeros and transferring it to your place. But what if that is all – what if computing power gets to the point where AI gets to the point where those boundaries are nonsense now? All encryption is instantaneously decoded.

[2:04:35] I think probably... [2:04:36] cryptography is [2:04:39] uh defense dominant in the limit i think like if you imagine um [2:04:45] Mature technology, I think it would be possible to encrypt. I mean if nothing else you could use like a one-time pad which would enable you to Encrypt things in a way that is unbreakable really? I [2:04:57] I feel like that's going to be a bottleneck. [2:05:00] I think once we start reading each other's minds That might be the first thing to go [2:05:05] It would be like the ultimate socialism. That's where you really probably would want encryption, right? If you're going to transmit thoughts to me. Oh, yeah. Like you don't want to sort of have a – Oh, you don't want some asshole constantly in your head, your next-door neighbor. Yeah. [2:05:17] just... [2:05:18] poking you and prodding you. But you would hope that along with this technology becomes like a general state of enlightenment that the human beings achieve where that's no longer the kind of behavior that we indulge in. [2:05:32] Which behavior do we no longer enjoy? Annoying each other, getting each other, stealing each other's money, that kind of stuff. But then, yeah, I guess we go back to this question of the utopian condition. Like if you – [2:05:44] So there are a lot of things that individually are bad. [2:05:48] Like lying, stealing, cheating, greed, excessive pride, like all kinds of disease, stubbing your knees bad, like... [2:06:00] Mm-hmm. [2:06:00] Yet, if you sort of imagine removing all of those things, then... [2:06:04] that changes the human condition quite profoundly. And to some people would feel kind of maybe or appear flavorless or sort of if there is...

[2:06:13] No tension, no conflict, no bruised ego. [2:06:20] It might still be good, but it does force us to sort of – I think it would be a rather fundamentally different thing that we would be metamorphosing into if we went all the way in that direction, which ultimately might be right, but it – [2:06:33] it would require us to kind of find new ways of realizing whatever values are imperfectly realized in the current world through conflict and competition and pain. Like some people get the motivation from painful failure. Yeah. And so if you get rid of the pain... [2:06:48] from failure, then you'd need some other [2:06:51] motivation, like some other thing that drives you on, which there could be, like maybe it's just a love of achievement and you feel kind of neutral or just less happy when you fail. But you would still need something that kind of preserves whatever structure it is that we think is valuable in the current human condition. Unless you go all the way to sort of radical hedonism and think the future is best if we were all just kind of floating together. [2:07:14] in some kind of drug-induced euphoria as blobs. [2:07:19] That's... [2:07:19] experienced immense pleasure, but I had no real experience. [2:07:23] texture in our experiences, didn't engage in activities and didn't interact with each other. Like there's like a philosophical view where ultimately pleasure is the only thing that matters and the minimization of suffering. So if that's your axiology, then it's relatively easy to see then how at technological maturity you would achieve a sort of optimal state. But if you have a more complex value system where maybe pleasure is one good thing, maybe really important,

[2:07:49] But there are also other things like appreciating beauty, true friendship, courage, achievement. And ideally, you'd want the future that includes all of these things. Then you need to do a little bit more sort of design work to figure out the way to combine them all in a meaningful way. But this all comes back to our idea of human meaning. What's important to humans, our finite 100 year lifespan. [2:08:14] adoption of this concept of meaning, but the black hole doesn't give a shit about human meaning. [2:08:20] and it's going to be around a lot longer than us and it's got a lot more power than us and it's doing a lot more change than us and we want to think that we're more important than black holes because we are us to us but to the universe is it is human meaning that important to the universe or is it just sort of a placeholder for what we'll ultimately become does it motivate us to continue to progress [2:08:50] Well, [2:08:51] So I think like at technological maturity, there are certainly forms of purpose that you could have. You could have artificial purpose. So this is when you basically set yourself a goal. [2:09:02] for the sake of having the goal and then doing the activity that... So maybe you set yourself the goal, I'm going to get this little white ball into a sequence of 18 holes. Thank you. [2:09:13] and [2:09:15] Not only that, but... [2:09:16] In order to achieve this goal, it's part of the goal that I'm only allowed to use this very inconvenient method. I can hit the ball with a club, right? I can't – much easier to just pick it up and put it in each hole successfully. But that doesn't count as being successful. So you could make up this goal –

[2:09:34] Pretty arbitrary. [2:09:36] Now, once you have that goal, then you now have reason to try hard, to concentrate, to perfect your swing, and you can play golf. So the goal enables you to do this activity of golf playing, which maybe you find fun or worthwhile or fun. [2:09:49] meaningful um and the reason why you find it meaningful is because it's difficult to do [2:09:55] Yeah, yeah. [2:09:56] And so the future would consist, I think, [2:10:00] If we succeed... [2:10:01] in a lot of game playing. [2:10:03] And you could certainly have these artificial purposes that you set yourself these goals that then give you a reason to engage in an activity. Now we're back in the world of Half-Life. Now we're back in a video game. Now we're also in the simulation. This is going to be your artificial goal. And you could imagine, I think maybe we shouldn't think of video games here, but it could be much like games we can't even imagine. [2:10:33] and little groups that work together to like come up with new ways of creatively. [2:10:39] And so... [2:10:41] Thank you. [2:10:41] In that broad sense of kind of things we do for their own sake, I think game playing could be. And it's like a lot of what children do, kind of for curiosity and spend a lot of their time playing games. And we might all be like kids again. What might be in shorter supply is... [2:10:56] natural purpose. Like... [2:10:58] purposes, which... [2:11:02] We don't just arbitrarily make up in order to have a purpose, but that's sort of given to us. So right now in the world...

[2:11:08] You might say – [2:11:10] you know, making a living... [2:11:12] It's not just an arbitrary purpose because there are real consequences if you fail. Like maybe eventually you get kicked out from your flat and then, you know, it's really cold and you get rained on and like horrible things happen. [2:11:26] So like similarly, if you... [2:11:28] like don't brush your teeth, eventually you will have tooth decay and there will be real consequences. [2:11:32] So there are various things that you have reason to do. [2:11:36] Because there are real negative consequences if you fail to do them and a lot of our lives is structured by these and [2:11:41] natural purposes. At a societal level, there's a whole bunch of things we need to do together, right? In this future world, maybe there would be many fewer of those natural purposes, because for any one of them, you could just ask the AI to sort it out. And so the artificial purposes would be a larger chunk. It's interesting to think, are there any natural purposes that would survive to technological maturity? [2:12:03] like any things that we still have sort of instrumental reasons that we need to do ourselves. [2:12:08] And I think there might be a few things [2:12:11] But they are more subtle. They might not strike us currently as very... [2:12:16] important but [2:12:18] And it's one of those things where... [2:12:21] Like, you know that during the day, if you're outside, you can't see the stars, right? [2:12:27] It's not because they're not there, like it's because like there's so much light that they are sort of blotted out. [2:12:34] At night... [2:12:36] Um, [2:12:36] when the stronger light from the sun is absent, you can see these fainter lights. I think similarly,

[2:12:42] in this future that might be [2:12:44] One's this sort of urgent screaming moral values of immediately pressing... [2:12:48] practical concern go away, you might be able to perceive a whole constellation of these more subtle values that we are blind to currently. So take the value of, I don't know, like some... [2:12:59] like honoring [2:13:00] your forebears. [2:13:02] So right now it doesn't seem, I mean, maybe it's nice sometimes to remember. [2:13:06] You're all right. [2:13:07] past parents or some historical hero who did something good that benefited [2:13:13] humanity, right? But it's not the main thing that you're... Maybe that would be a bigger thing if there was nothing else you needed to do. Maybe you could actually spend serious time or spiritual quests. Even for people who are very religious, a lot of their actual waking hours are spent on [2:13:30] random other things, doing their laundry, right? Like driving to work, like all kinds of stuff. Like if all of that was automated, you could imagine spending more time, [2:13:38] on trying to align yourself, orient yourself to this higher being and trying to be in communication with them. Like maybe aesthetic... [2:13:48] values. Like there may be [2:13:50] some things that would just be kind of nice and cool if the world were like that. We don't really have time to worry so much about them now, but if there was nothing else on the agenda... [2:13:58] like coming together in a way that upholds some tradition in a beautiful original way that still is true to the original spirit. [2:14:07] together with other people and enacting some ceremony. Like maybe those things would start to fill more of our time in conjunction with this game playing.

[2:14:15] Um, [2:14:16] Um... [2:14:17] And there might be many other... [2:14:19] of these kind of subtler values that would start to shape [2:14:23] what people were doing. [2:14:25] Thank you. [2:14:26] Yeah, and... [2:14:28] Ultimately, who knows? Yes. It's... [2:14:32] It's very interesting, and it's very open-ended, and we really don't know what's going to happen, but we're probably going to see it. [2:14:39] We're probably going to see the strangest... [2:14:41] thing that humans have ever had a possibility to... [2:14:46] Yeah, and in the end, I guess it's trustful. Yeah. [2:14:50] Well, [2:14:52] I mean, these conversations are always fascinating. And who knows? Let's do another one in a few years and see how off we are. Come back in four years. All right? If we have four years. If we have four years. If you can, if you're allowed to travel in four years, come back and let's see how wrong we were. Make an update. Well, thank you very much. I really appreciate you coming in here. It's great to see you again. It's fun. It's always fun. Okay. All right. Think about it, kids. Bye, everybody. [2:15:22] Thank you.