The Limits of Computing

[Duxwing]

Dear Forum,

@Alice? made me realize that we need to discuss something other than politics, religion, and love, and @Lyra pointed out that our discussions often rest upon unstated axioms, so I'll try to see if I can make a post that doesn't do either of these things.

Will the presently ever-increasing power of computers, as defined by the density of transistors on a silicon chip, ever reach a limit, and if so, then how and why?

-Duxwing

 

[ProxyAmenRa]

Will the presently ever-increasing power of computers ever reach a limit, and if so, then how and why?

-Duxwing

Do you mean power vs. density of cpus or just a powerful computer? Single thread power? Multi-thread power? If your software is capable of utilizing multiple threads, you can simply link n number of threads together like a super computer. In such a circumstance the only limiter is the amount of energy and matter needed to create and run the thing.

 

[Duxwing]

Do you mean power vs. density of cpus or just a powerful computer? Single thread power? Multi-thread power? If your software is capable of utilizing multiple threads, you can simply link n number of threads together like a super computer. In such a circumstance the only limiter is the amount of energy and matter needed to create and run the thing.

Edited to acknowledge your point.

-Duxwing

 

[SpaceYeti]

Dear Forum,

@Alice? made me realize that we need to discuss something other than politics, religion, and love, and @Lyra pointed out that our discussions often rest upon unstated axioms, so I'll try to see if I can make a post that doesn't do either of these things.

Will the presently ever-increasing power of computers, as defined by the density of transistors on a silicon chip, ever reach a limit, and if so, then how and why?

-Duxwing

Of course technology of any kind has a cap in raw capability to size/space, because it exists in a universe which operates on objective processes, and those laws cannot be changed just because we want more power. It's like making hydrolic lifts. They're basically at the prime of their technological existence. If we tried to make them smaller and just as strong, well, we couldn't. It's the same thing on the small scale of computer chips, except I don't know enough about those to predict when they've met their best efficiency. There will be a point in computers where the chip's raw ability is maxed out for it's size. In those days, invention will be how they're used, not making them more powerful (because they're already as powerful as they possibly can be).

 

[Architect]

Present silicon will reach it's final limit around 9 or 12 nm. We're already at the point where quantum effects must be accounted for when designing chips, by then they will dominate. As usual for this point of the technology curve we are developing technology extenders that will keep present silicon alive for some time longer. The main candidate appears to be 3D transistors. A friend of mine is playing a key part in that. Interestingly the big problem is not the technology, but how to get different companies to work together enough to be able to interconnect them. At any rate 3D transistors offer great speed advances with much better thermal performance.

Once we get to the limit we'll continue squeezing more performance out through this and other various methods. Then the replacement paradigm will come along, just as it did when we went from mechanical to tube and tube to silicon. Graphene seems promising, or perhaps biocomputing.

As for the ultimate limits this has been estimated fairly accurately. It's a physics problem, not too difficult. For full details consult The Singularity is Near by Ray Kurzweil. All of these other points are spelled out there too. The actual computing limits are quite large, it should take a long time to get there, and will probably move beyond our Solar System by then to get more computing power.

Anyhow read the book, it's all spelled out there in an accessible way. Some interesting factoids. By 2050 or thereabouts (I forget the precise date) our desktop computers will be powerful enough to compute every thought of every human being in our entire history.

PS It appears that neither parallelization or quantum computing will come to the rescue, though they both are powerful techniques. The problem is that they are useful for a non universal class of problems.

However parallelization will nicely solve the problem of mirroring the human brain in silicon. I think the answer will be HHMMs (Hierarchical Hidden Markov Models) running on graphics CPU's as discrete Boltzmann machines.

 

[Duxwing]

Present silicon will reach it's final limit around 9 or 12 nm. We're already at the point where quantum effects must be accounted for when designing chips, by then they will dominate. As usual for this point of the technology curve we are developing technology extenders that will keep present silicon alive for some time longer. The main candidate appears to be 3D transistors. A friend of mine is playing a key part in that. Interestingly the big problem is not the technology, but how to get different companies to work together enough to be able to interconnect them. At any rate 3D transistors offer great speed advances with much better thermal performance.

The ENTJs will get it together eventually... I hope.

Once we get to the limit we'll continue squeezing more performance out through this and other various methods. Then the replacement paradigm will come along, just as it did when we went from mechanical to tube and tube to silicon. Graphene seems promising, or perhaps biocomputing.

Yep.

As for the ultimate limits this has been estimated fairly accurately. It's a physics problem, not too difficult. For full details consult The Singularity is Near by Ray Kurzweil. All of these other points are spelled out there too. The actual computing limits are quite large, it should take a long time to get there, and will probably move beyond our Solar System by then to get more computing power.

Humans: monkeys that like math.

Anyhow read the book, it's all spelled out there in an accessible way. Some interesting factoids. By 2050 or thereabouts (I forget the precise date) our desktop computers will be powerful enough to compute every thought of every human being in our entire history.

*starts drooling and licking his lips* So much power!

PS It appears that neither parallelization or quantum computing will come to the rescue, though they both are powerful techniques. The problem is that they are useful for a non universal class of problems.

Maybe we can make all other problems into those types of problems?

However parallelization will nicely solve the problem of mirroring the human brain in silicon. I think the answer will be HHMMs (Hierarchical Hidden Markov Models) running on graphics CPU's as discrete Boltzmann machines.

Hamanahamanahamana, that last sentence was almost entirely made of awesome. What's a Boltzmann machine?

-Duxwing

 

[Architect]

Hamanahamanahamana, that last sentence was almost entirely made of awesome. What's a Boltzmann machine?

Actually I don't know even though I believe the technique came from Physics, this is very recent results. The point is somebody figured out how to massively parallalize ANN's so they could be run on these wonderful GPU's we have. Forget multi core CPU's, it's the gamers that have demanded supercomputers in their GPU's.

I've been meaning to learn up on it if I can find any materials.

 

[Duxwing]

Actually I don't know even though I believe the technique came from Physics, this is very recent results. The point is somebody figured out how to massively parallalize ANN's so they could be run on these wonderful GPU's we have.

What's an ANN?

Forget multi core CPU's, it's the gamers that have demanded supercomputers in their GPU's.

That's too cool!

-Duxwing

 

[Architect]

What's an ANN?

Artificial Neural Net. They languished for decades because we didn't have computers fast enough to run them. Now they're one of our most powerful techniques.

 

[ProxyAmenRa]

Artificial Neural Net. They languished for decades because we didn't have computers fast enough to run them. Now they're one of our most powerful techniques.

And they're also terrible at extrapolating. And they encourage the average person to be lazy.

 

[scorpiomover]

And they encourage the average person to be lazy.

I thought that was the goal, to make things easier and easier, so people have to expend less and less physical and mental effort.

 

[ProxyAmenRa]

I thought that was the goal, to make things easier and easier, so people have to expend less and less physical and mental effort.

The goal is to make the domain expert's work easier so time can be better allocated.

The average person sees ANN as this wonderful thing that can do anything. The black box which produces magic. They fail to realize that ANNs still need to be tailored to the system they're trying to model/map/control. This lack of awareness lead them to produce rubbish. Unlike the domain expert, everyone else still need to gain knowledge and wisdom through experience and inquiry.

And you still need to read chapters 1 to 4 of 'Man, Economy and State'.

 

[scorpiomover]

The goal is to make the domain expert's work easier so time can be better allocated.

No. That's the expert's goal. It's only relevant, if he is writing an ANN for himself, that is paid for, by himself, and will never be seen or used by anyone else.

The average person sees ANN as this wonderful thing that can do anything. The black box which produces magic.

That's the way most people see computers, especially those who love technology. An ANN is just a subset of computers. Those who think computers are wonderful things that can do anything, think the same of ANNs. Those who think computers are not wonderful, and/or are limited, think the same of ANNs.

They fail to realize that ANNs still need to be tailored to the system they're trying to model/map/control. This lack of awareness lead them to produce rubbish. Unlike the domain expert, everyone else still need to gain knowledge and wisdom through experience and inquiry.

Yes. It does strike me that what I usually read people saying IT, is usually full of irony. I've usually been involved with people and projects that were attempting something similar, several years previously, and know the results of those projects, and the analysis of their results, and the similarities are usually enough that the analysis would still hold true. Yet the claims of the present are usually being stated as if those people could not possibly have any knowledge of those analyses.

Put another way, most of my job in IT, was about explaining to people who think that they know about computers, why the things that they want but don't need, are usually totally unrealistic, and why the things that they need but think cannot be done, are usually really, really easy to do.

And you still need to read chapters 1 to 4 of 'Man, Economy and State'.

That's not relevant to the topic. But, as it happens, I have a lot of experience and carried out a lot of inquiry in that area. It was one of the subjects that I happened to have a lot of exposure to, and in which my INTP-ness endeavoured to analyse for patterns, and a great many, I found.

 

[Architect]

The goal is to make the domain expert's work easier so time can be better allocated.

The average person sees ANN as this wonderful thing that can do anything. The black box which produces magic. They fail to realize that ANNs still need to be tailored to the system they're trying to model/map/control. This lack of awareness lead them to produce rubbish. Unlike the domain expert, everyone else still need to gain knowledge and wisdom through experience and inquiry.

And you still need to read chapters 1 to 4 of 'Man, Economy and State'.

You're hinting at expert systems, but actually that's been tried a bunch of times. The problems with that general approach is that they are expensive/difficult, experts don't know how to quantify how they know what they know, and ultimately they don't work well, usually spectacularly bad. Few people do them anymore, the big exception is Wolfram Alpha, and when's the last time you used that? Now when is the last time you used any of the Google services such as search, street view or translate?

Watson, arguably our best A.I to date, took an idea I've head for a decade which is they grabbed every A.I they could get their hands on and threw a statistical decision maker on top of it.

Kurzweil is of the opinion that HHMMs (Hierarchical Hidden Markov Models) are the solution, and I'm inclined to agree.

 

[ProxyAmenRa]

What I mean by the 'domain expert' is the guy who is extremely knowledgeable and experienced. You know? Like my old boss. His time is worth more than my time.

YKurzweil is of the opinion that HHMMs (Hierarchical Hidden Markov Models) are the solution, and I'm inclined to agree.

I have seen some demos on the subject. Definitely interesting stuff.