To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.
Great Social Content — Consistent with other areas of online marketing, content reigns supreme when it comes to social media marketing. Make sure you post regularly and offer truly valuable information that your ideal customers will find helpful and interesting. The content that you share on your social networks can include social media images, videos, infographics, how-to guides and more.
Social media can be used not only as public relations and direct marketing tools but also as communication channels targeting very specific audiences with social media influencers and social media personalities and as effective customer engagement tools. Technologies predating social media, such as broadcast TV and newspapers can also provide advertisers with a fairly targeted audience, given that an ad placed during a sports game broadcast or in the sports section of a newspaper is likely to be read by sports fans. However, social media websites can target niche markets even more precisely. Using digital tools such as Google Adsense, advertisers can target their ads to very specific demographics, such as people who are interested in social entrepreneurship, political activism associated with a particular political party, or video gaming. Google Adsense does this by looking for keywords in social media user's online posts and comments. It would be hard for a TV station or paper-based newspaper to provide ads that are this targeted (though not impossible, as can be seen with "special issue" sections on niche issues, which newspapers can use to sell targeted ads).
Link text is the visible text inside a link. This text tells users and Google something about the page you're linking to. Links on your page may be internal—pointing to other pages on your site—or external—leading to content on other sites. In either of these cases, the better your anchor text is, the easier it is for users to navigate and for Google to understand what the page you're linking to is about.
Since social media marketing first came to be, strategists and markets have been getting smarter and more careful with the way they go about collecting information and distributing advertisements. With the presence of data collecting companies, there is no longer a need to target specific audiences. This can be seen as a large ethical gray area. For many users, this is a breach of privacy, but there are no laws that prevent these companies from using the information provided on their websites. Companies like Equifax, Inc., TransUnion Corp, and LexisNexis Group thrive on collecting and sharing personal information of social media users. In 2012, Facebook purchased information from 70 million households from a third party company called Datalogix. Facebook later revealed that they purchased the information in order to create a more efficient advertising service.
On April 24, 2012 many started to see that Google has started to penalize companies that are buying links for the purpose of passing off the rank. The Google Update was called Penguin. Since then, there have been several different Penguin/Panda updates rolled out by Google. SEM has, however, nothing to do with link buying and focuses on organic SEO and PPC management. As of October 20, 2014 Google has released three official revisions of their Penguin Update.
Expertise and authoritativeness of a site increases its quality. Be sure that content on your site is created or edited by people with expertise in the topic. For example, providing expert or experienced sources can help users understand articles’ expertise. Representing well-established consensus in pages on scientific topics is a good practice if such consensus exists.
Small businesses also use social networking sites to develop their own market research on new products and services. By encouraging their customers to give feedback on new product ideas, businesses can gain valuable insights on whether a product may be accepted by their target market enough to merit full production, or not. In addition, customers will feel the company has engaged them in the process of co-creation—the process in which the business uses customer feedback to create or modify a product or service the filling a need of the target market. Such feedback can present in various forms, such as surveys, contests, polls, etc.
Search engine optimization consultants expanded their offerings to help businesses learn about and use the advertising opportunities offered by search engines, and new agencies focusing primarily upon marketing and advertising through search engines emerged. The term "search engine marketing" was popularized by Danny Sullivan in 2001 to cover the spectrum of activities involved in performing SEO, managing paid listings at the search engines, submitting sites to directories, and developing online marketing strategies for businesses, organizations, and individuals.
While most search engine companies try to keep their processes a secret, their criteria for high spots on SERPs isn't a complete mystery. Search engines are successful only if they provide a user links to the best Web sites related to the user's search terms. If your site is the best skydiving resource on the Web, it benefits search engines to list the site high up on their SERPs. You just have to find a way to show search engines that your site belongs at the top of the heap. That's where search engine optimization (SEO) comes in -- it's a collection of techniques a webmaster can use to improve his or her site's SERP position.
The platform of social media is another channel or site that business' and brands must seek to influence the content of. In contrast with pre-Internet marketing, such as TV ads and newspaper ads, in which the marketer controlled all aspects of the ad, with social media, users are free to post comments right below an online ad or an online post by a company about its product. Companies are increasing using their social media strategy as part of their traditional marketing effort using magazines, newspapers, radio advertisements, television advertisements. Since in the 2010s, media consumers are often using multiple platforms at the same time (e.g., surfing the Internet on a tablet while watching a streaming TV show), marketing content needs to be consistent across all platforms, whether traditional or new media. Heath (2006) wrote about the extent of attention businesses should give to their social media sites. It is about finding a balance between frequently posting but not over posting. There is a lot more attention to be paid towards social media sites because people need updates to gain brand recognition. Therefore, a lot more content is need and this can often be unplanned content.
Social media often feeds into the discovery of new content such as news stories, and “discovery” is a search activity. Social media can also help build links that in turn support into SEO efforts. Many people also perform searches at social media sites to find social media content. Social connections may also impact the relevancy of some search results, either within a social media network or at a ‘mainstream’ search engine.
In December 2009, Google announced it would be using the web search history of all its users in order to populate search results. On June 8, 2010 a new web indexing system called Google Caffeine was announced. Designed to allow users to find news results, forum posts and other content much sooner after publishing than before, Google caffeine was a change to the way Google updated its index in order to make things show up quicker on Google than before. According to Carrie Grimes, the software engineer who announced Caffeine for Google, "Caffeine provides 50 percent fresher results for web searches than our last index..." Google Instant, real-time-search, was introduced in late 2010 in an attempt to make search results more timely and relevant. Historically site administrators have spent months or even years optimizing a website to increase search rankings. With the growth in popularity of social media sites and blogs the leading engines made changes to their algorithms to allow fresh content to rank quickly within the search results.