I'm Not A Mathematician But...

This is from Wikipedia -

Google runs over one million servers in data centers around the world,[14] and processes over one billion search requests[15] and twenty petabytes of user-generated data every day.[16][17][18]

According to a story on Cnet a few months ago, each server has two 4 core CPU's. They are the biggest supercomputer in the world. The data processing you describe is just regular database operations for the most part and is no strain on their systems at all.

When's the last time your searches took more than a fraction of a second? All of those million plus servers are connected. And with terabytes of memory their whole bloody index can be in RAM.

The stuff we see is trivial. Googles computing focus has been AI.

ALSO, Google preprocesses everything. They PROBaBLY even have various keyword searches already preprocessed. I did something like what google does. Mine was better in some ways, and was running on ONE system, but mine could take a few seconds. And I was only going through a few million sites. Of course, secondary searches were FAR faster because of cache. If I simply SEARCHED every site in the database to do the searches, it would take several MINUTES!

Steve

Lawrh - are there any exact specs listed in those articles? clock speed, bus speed, type & amount of ram, etc? I've never had searches come back in less than a few seconds, true, but I have had gmail stall out on me, chat lag & break, blogger service been unavailable, etc. Google does a lot of different types of processing. Also, like you said they have millions of computers worldwide - the idea is to ascertain if they have the capability to process that other information, which is probably just as much if not more data than what results from their daily search queries, on top of what they're already doing out in the open.

seasoned - sounds like you've got a really good idea of how their search engine functions - how do you think your machine would react to multiple queries at the same time? How much data does the processor have to move in each function? Could you determine a standard size based on database size & search string? could give more insight into calculating exactly what google machines are capable of...

I'm sorry if I offended either of you guys - didn't intend to cause a fuss with the post.

It's just an interesting fact that we don't know what they're capable of - there's no references to actual hardware, exactly how many datacenters, etc in any of those cnet articles, they were more like PR articles for Google - yet the word fear is used in the same sentence with Google, even in the absence of fact. It's superstition. It's paranoia. It's unhealthy & counter-productive. I think that's what I was getting at with my original post.

Talking about hardware and the measure of its capabilities is somewhat an art & science. At best, we could come up with a theoretical "how this system should operate under laboratory conditions" measure of its' capability. Everything about any given computer system is known, at least on paper. There are outside influences, like power fluctuation and temperature, but hardware is made to work within a certain range & the software's code is known and programmed to work in a specific way. I will admit that there's no finite way to pin point exactly how a system will perform at any given moment, but again, I think the point was to provoke thought about whether we should be so quick to assume that Google is really plotting our ultimate demise - if we could prove that in theory they were mortgaged to the data hilt, so to speak, or somewhere close to it (because we know they process over 24 petabytes of data every day), even if only theoretically, we could say that we could rule out Google expending extra power just to analyze our activity like big brother.

And I am speaking about their capabilities at the server, not their bandwith issues. I have had chat lag excessively only to tell me moments later that it's sorry for my inconvenience, chat is having technical difficulties & Google is working on it (lags & breaks) - blogger gives me "service unavailable" or "server error", etc. that does show an deficiency somewhere in their system, not their bandwith. To complicate things, I've had this happening in the middle of the morning, hardly peak surfing time.

Again, like I said, I didn't mean to upset you guys, but rather to ask you to think logically - what do you think Google is capable of? Without proof, is it more likely that theories and superstitions are thrown around, or facts? Since there's no proof, there are no facts.... superstitions are what these ideas are, not fact.

LOL - have you ever been to one of their data storage complexes? How about one of their technician centers. If you were to put all of them in the same place, google owns a pretty nice size city.

I like their complex out in the Dalles, Oregon - use goats to mow their lawns. Employees take their pets to work and they have a pretty cush gym and campus. Of course, the goats aren't eating the main greensward - they just munch all those voluminous acres that google owns with the complex.

For a corporation the size and strength of Google, I really don't think power OR time is much of a concern.

Actually, Lawrence - that is the type of info on their Annual Report and anyone can request a copy, stockholder or not -- oh, library has em, too.

Actually there's nothing on paper that can compare to a walk through of one of their facilities. If 3 or 4 more other Google complexes were put beside the one in the Dalles, they would replace the whole town - just a few of them at that size. If you think you can comprehend what those complexes are like on the inside........I'm betting a tour would wow you........scared me crapless. I've seen Microsoft and Yahoo data storage facilities from the outside (the security it tight, you aren't just going to walk in the door) and they both would fit in one google complex with a lot of room left over for something else small like BOA or something else insignificant. LOL.

And......I'm still most impressed with the goats. LOL.

lawrence - post the link to your source and the exact number of computers and their specifications in this thread.

The entire undertone of what you say is that it's a fact that Google monitors every inch of the webspace visible to them (which I have no doubt they do) and then uses that information to some end - which I don't necessarily believe that they do other than to know generic usage statistics. You talked about Google being one of the largest researchers of AI & that's a reason to fear them - watch your words, what are they doing with their AI technology? Why should there be fear? You're implying something there with no proof.

As for whether I'm experiencing trouble with a local datacenter - it goes to prove my point that their servers experience load & it can cause problems - if it happened to my local datacenter, then why not others? and if it could happen to one, then what does that say about the capabilities of the other datacenters.

I'm serious, you don't have any concrete facts & yet you're implying that Google can do this that or the other, when you don't know. That's superstition - that's a belief in something other than fact. Prove me wrong?

no, didn't skip sal's post - there's no information available to the public - you're both wrong about that. I actually contacted google directly, they said the information I requested wasn't available to the public - not to me or their shareholders. It's called trade secrets.

You're still wrong about the server problems as well - if I can connect to google in order for them to give me a server error message or tell me that there's a problem with their chat service, there's no problem with the connection - server error is server error. I'm not sure you know as much as you claim to. In regards to distributed computing, I think you've completely ignored what I've said in previous posts - I understand that the workload is spread out over a multitude of systems, but there are still limitations - if you have two computers sharing the load, they can do the work of one much more efficiently, but they can't do the work of four computers.

and I read everything regarding google on cnet - there's no information there other than some vague reports that publicized something that was going on with Google at the time of the report.

Lawrence, you're wrong.

Also, you've not addressed what I've said about you claiming we should fear Google. You keep mentioning AI & I see your articles that you've referenced here - I'll quote what Larry Page says:

"Artificial Intelligence would be the ultimate version of Google. So, with the ultimate search engine it would understand everything on the web - it would understand exactly what you wanted, and it would give you the right thing. And that's obviously Artificial Intelligence, to answer any question, basically, because almost everything is on the web, right?

So, we're nowhere near doing that now, however we can get incrimentally closer, and that's basically what we work on & that's tremendously interesting from an intellectual standpoint."

The article you posted to there quotes the video, then goes to break down those two paragraphs - which took all of a minute for lary page to spit out, then shows a couple videos later, about a minute in length each where larry page again says "we'd like to have a search engine that can give you exactly what you wanted, using ai" and another where he addresses a conference & talks about what ai will be, and I quote - "when ai happens, I think it will be a lot of computation, and no such clever blackboard/whiteboard kinds of stuff or other algoriths, but a lot of computation" - before that he talks about his dad trying to do AI on a commodore - he also says that they have "some people at Google working on Artificial Intelligence" and then he says the same thing from the two previous videos, "to do a perfect job of search, you could ask any query & it would give you the perfect answer" and then "we're lucky enough to be working incrementally closer to that, but again very very few people are working on this" - then he goes on to talk about climate change - such as the weather - larry page wants to change the climate, or so he says. So he was a speaker at a conference about climate change a took a second to plug his search engine.

Lawrence, did you read the article & really listen to what larry said? He literally says these things

- AI is not being implemented, they're working on it
- There are very few people working on AI
- They're making small, incremental steps (it's going very slowly, at least that's what he says more than three times in the videos that you've offered)
- AI is being developed to improve search, nothing else - not used to analyze any other data at Google.

So why do you keep telling me about AI? Not only does the material you've referenced prove that (and that last video was in 2007) Google most likely still doesn't have any kind of AI implemented anywhere in their system that interacts with the public, but it's taking them a long time - the first video is from 2000 - the second 2006 and the third 2007 - that's seven years that Larry Page has been saying the exact same thing. Literally, it's a script he has well memorized because he repeats it word for word three times in interviews or speeches that he's given over a seven year span.

I think that there's a lot of mysticism in your own mind about Google & what they're doing & capable of. I really think that you're overlooking a lot of facts, facts that you've tried to present here yourself - there was very little substance to that page or videos on that page, as was the case with the CNET articles and there's absolutely no hard facts about what kind of technology Google does have - both you and sal were wrong, I got it straight from the horse. If it was public information I could get at my local library, why wouldn't they tell me in an email? I think I'll have to go to my public library and get that information tommorrow, then I'll email it to Google & ask them why they would withhold the information that I can so easily produce in my own home town. Seriously, that's what you're telling me - am I wrong?

Wait a minute here, you're calling me naive but you've not proven anything yourself - again in your last response you've stated that AI is Google's main focus, but cite videos in which Larry Page actually says that "very few people are working on this (AI)".

Google: Server Efficiency Needs New Recipe

There's your CNET article. It makes a reference to Google having hundreds of thousands of servers, but in the context of the power consumption & doesn't give any detailed information. There's a link within that article to the CNET article on their data centers as well - a 250 word article that speculates on the location of their data centers, but no facts.

I'm not trying to hold on to my naivete, I'm just responding to your posts that make claims they don't support. Other people that read these posts and don't know that much about how Google operates are going to read your words -

After making that statement on a public forum, you then cite articles that don't support the statement. And I think what's worse is that you've chosen to tell the entire Warrior Forum that they should be scared of Google.

There's a lot of misinformation in IM that opens the doors for a lot of scammers - I think that's why I continue to make my argument. Anyone on this board could capitalize on what you've said by referencing this thread & then writing a poorly constructed eBook, selling it to the people who believe what you've said.

As for mysticism, that's what I'm left to assume is at play here, because it's not fact. It never has been & you've proven that repeatedly.

This demonstrates a quality often expressed in this forum. Someone does not understand something (corporate level computing) or cannot do something (research) therefore declares that it is impossible or does not exist. Everything Sal, Steve and I have said is true. G even recently signed a contract with NASA to take over some of their data processing. Maybe NASA should have checked with you first.

steve,

that occurred to me as well. I actually found a good article on wikipedia that showed their setup in 1999, the servers they were running then were something like dual 300mhz intel pentium II chips. Here's an article:

Coding Horror: Google Hardware Circa 1999

That just puts another spin on what their actual capabilites are - if they actually ran old hardware like that then the sentence - Google having hundreds of thousands of servers - takes on a whole new meaning.

Lawrh,

You're not proving anything. You're just repeating yourself & not supporting your argument. The idea behind my entire post was that nothing can be definitely proven about what Google does or does not do, because we don't know any specifics about their hardware. You still haven't proven what they do or don't have & everything that you have said, you've proven yourself wrong about.

No, you've misunderstood completely. That's never been my argument. I speculated on whether they used the information they collect outside of the google bot (such as adsense, analytics, etc) to manipulate the rankings.

I then went on to state that there's no hard evidence one way or another about it. I did speculate about how Google was multifaceted and that it would require a lot of computing power to do what they do on a daily basis. Then I pointed out that we really don't have any definite information about what kind of power Google really has behind the scenes.

I also admitted in the original post as well as throughout the other posts that Google could do whatever they wanted - they have the money to buy the machines to do whatever they want.

I've never made any claims that Google definitely could or could not do something or that there was any kind of weakness in their system. What I've said is that no one knows any hard facts about Google's capabilities or what they actually do. In the absence of fact, however, people make claims that Google in fact does do one thing or another.

That's what I've been talking about the whole time - we don't know anything about Google, but a lot of people make claims & pass them off as fact.