We’re currently living through the fourth industrial revolution, often called the Information Age. The reason for this clear distinction is that because currently, a vast majority of the global population uses the world wide web, or more accurately, search engines, to access information. However, even with such widespread reliance and use, not many of us actually know what a search engine is or how it works.
Simply put, a search engine is software created to retrieve specific information. More specifically, internet search engines are web services which have been developed to find and display information from other websites.
Google, the most popular search engine in the world
The History of Search Engines
Today, search engines have become synonymous with the internet and web browsers. This is partly a result of Google Chrome including the functionality of a search engine in the web address bar. However, a browser and a search engine serve different purposes.
‘Archie’ was the first of many search engines and was developed in 1989 by Alan Emtage as a university project at McGill University. Alan Emtage was part of the university’s Information Technology department and created Archie to assist him in gathering software data for other faculty members and students.
Bill Heelan and Peter Deutsch further developed the code that Emtage used to create the first search engine to allow other people to also log on and use it. Archie was then debuted to the rest of the world on September 10, 1990. However, this interface and the results were very different from the search engines that we use today.
Archie was mainly an index of File Transfer Protocol (FTP) sites and was used to transfer documents between computers. There were no creative or interactive graphics as there are now, as it only had text-based page content. It wasn’t programmed to use natural language keywords, so search words had to be limited to a few keywords.
As a nod to the popular comic book, the subsequent search engines were named Jughead and Veronica. These search engines were established through the introduction of the Gopher communication protocol. This system made it possible to distribute, search for, and obtain information on the internet. This gave rise to the many other search engines that exist today.
A screenshot of Archie, the first search engine.
A Timeline of the Development of Search Engines:
- 1989: Alan Emtage developed the first public search engine.
- 1994: David Filo and Jerry Yang introduced Yahoo! as an internet portal to their favourite websites.
- 1995: AltaVista created the first natural language search engine that accepted questions written in spoken language.
- 1998: Larry Page and Sergey Brin founded Google.
- 2000: Google AdWords, the first ever self-serve advertising platform modelled on ‘pay-per-click’ (PPC) was launched.
- 2009: Live/MSN search was relaunched as Microsoft Bing.
The Importance of a Web Search Engine
The average person is exposed to a lot of information every single day. Advertisements at every turn, newspaper headlines, social media frenzies, text messages, etc., are bombarding us 24/7. This makes it increasingly difficult to remember all relevant facts and we often need the assistance of various tools like search engines.
We have search engines that manage our emails, remind us of our schedules, update our calendars, and assist us by getting information on almost anything we need.
Most of the major search engines now offer alternative services such as email.
The Usefulness of Search Engines
This task is more important than we realise, and web search engines help us:
- Find information quickly – There are more web pages on the internet than there are people on Earth. Search engine results help us find information quickly by filtering out everything irrelevant.
- Get us answers from high-quality websites – Search engines use keywords and search queries to sift through pages of information about a variety of topics. This way, they provide us with answers by showing the most relevant and informative pages first.
- Separate the wheat from the marketing chaff – Search engines are not flooded with advertisements and often make it clear which results are paid for as marketing. This helps us avoid sites marketing products and helps us get genuine search results.
- Make decisions about brands and products – Search engines such as Google and Yahoo have become influential in how we perceive products or brands. They allow a multitude of people to share their opinions and make it easy to get honest feedback and/or information from the brands themselves.
- Become more self-reliant – The ease and accessibility of information on search engines have made it possible for people to become more confident in themselves and gain more knowledge. People can easily search for advice, translate languages, purchase items, and so much more.
How do Search Engines Work?
Most search engines work in the same or similar ways to assist you in getting the search results you want. This can be broken down into three stages, namely: web crawling, indexing, and ranking.
Web Crawling
This first step is the process of finding information. Search engines make use of software known as ‘web crawlers’ or ‘search engine spiders’ to search for available information on the internet. These crawlers locate and investigate internet servers that act as hosts for various websites.
Once a list of servers is created, a total number of hosted websites is determined. During this process, the crawlers also determine the number of pages on each website and the content that the websites contain. This content can include text, videos, audio, and images.
Lastly, if there are any hyperlinks on the website, the crawler will investigate those too. These links could be internal or external, leading to the discovery of more web pages, where the process repeats.
Indexing
This second part of the process is when the organisation of various websites takes place.
The information gathered during the ‘crawling’ phase is now sorted out, organised into a hierarchical order, and stored away for later processing. This processing will be done through the use of algorithms that evaluate the relevance of each web page.
Algorithms also prepare the pages for ranking, and allow for effective presentation and access to the user. During this indexing phase, only the information deemed most important by the algorithms will be stored by the search engine.
Search engines use bots to crawl the internet and feed back data to the algorithms.
Ranking
After the information has been organised during the indexing process, the search engine results pages (SERPs) are then ranked. This ranking serves to evaluate and provide search results that will accurately respond to search queries.
With the increasing density of information on the internet, these algorithms have become more complex. Due to this, an entire industry called search engine optimisation has come to fruition, working with businesses to increase the organic position of their webpages in the search results.
How Do Search Engines Rank Results?
One search query can return millions of web pages. However, not all the information on these pages is useful to the user, as it may be inaccurate, spammy, or insufficient.
The algorithms that rank web page content assess these pages according to certain factors. Interestingly, these factors are biased toward user engagement.
Search engine algorithms want to give results that users will respond to by visiting the page. When users spend time on a particular website, this engagement sends a signal of relevance to the search engine. Ultimately, the factors are based on what gets the highest levels of user engagement. These include:
- Keywords: The words people use in their search queries determine what information will be presented. Search engines place high importance on web pages that have the user’s keywords in more notable positions throughout the text, such as the headings.
- Webpage Content: High-quality content is determined through the evaluation of the length of the content and other qualities and dimensions of the web page.
- Inbound Links: These are also known as ‘backlinks’, and are links or mentions from one website to a different web page. These act as an acknowledgement or approval of the other website’s credibility.
- User Information: Many search engines use people’s personal information, such as one’s location and search history, to provide results that would be especially relevant to them.
- Offensive language: Search engines are legally obligated to filter out information that is illegal or highly offensive to various groups.
The Largest Search Engines
There is a multitude of search engines that exist today to assist us in retrieving information all over the world. Out of this large variety, there are 5 that are most frequently used.
I’m sure it’s no surprise that Google is the largest and most popular search engine in the world. This search engine gained so much popularity that it’s often used as a verb that means “to search”.
Google has 83.84% of the search engine market share in the world. Since its conception in the 90s, it has managed to retain its dominance in the industry. This is by constantly incorporating new ideas into its browser, being consistent in delivering relevant content, and creatively establishing exclusive agreements with various device manufacturers.
These strategies have resulted in the funnelling of about 60% of internet searches straight to Google.
The headquarters of Google, known as Googleplex, Mountain View, California
Bing
Bing is Microsoft’s search engine and holds 8.88% of the world’s web search engine market share. Since its launch in 2009, it has become the second largest search engine in the world, receiving an average of 1.3 billion visits a month.
Bing also tries to incorporate fresh ideas and consistent updates, such as the interesting photography and news stories that are featured on the homepage. Additionally, Microsoft and Yahoo! made a deal whereby Bing would power Yahoo!’s search engine.
Baidu
Baidu’s main customer base is made up of mainly Chinese users, and it’s currently the third largest search engine in the world. Baidu has a global market share of 0.64% and heavily dominates the Chinese market with 74.73% as of 2019.
This search engine was founded in China in 2000, with its headquarters in Beijing. Since then, it has expanded to be one of the largest tech companies in the world, specialising in artificial intelligence and providing internet service.
Yahoo!
Yahoo! is the world’s third largest search engine with a 2.55% global market share. This is unfortunate considering the early start that it got in 1994.
Yahoo! Mail and other internet services raised the company valuation to $125 billion in 200o. However, since then, it has lost many users to Google and, in 2017, was sold to Verizon. As the search engine is powered by Bing, you may find very similar results when using either of them.
Yandex
Yandex has a global market share of 1.67% but has a very large presence in Russia as the most popular search engine. The name “Yandex” has been used since 1993 and is short for “Yet Another Indexer”.
It holds a market share of 55% in Russia, with Google nearby in second place. This search engine’s popularity also extends to Ukraine, Belarus, Kazakhstan, Uzbekistan, and Turkey. This is largely because of its ability to pick up Russian intonation in search queries.
Yandex also provides over 70 other services, including a cloud storage tool similar to Google Drive. This is called Yandex Disk.
Yandex is the largest search engine in Russia
