How to find almost all directories?
There are three different ways of building systematically your own link directory (or social bookmarking site or article directory) database — all of them is based on finding certain footprints or texts present on only the kind of sites you are looking for by scraping search engine result pages.
Finding sites which call themselves directories
It might be rather ineffective on the English web due to the enormous volume of (useless, paid or otherwise not that useful) directories where a search engine “web directory” or “link directory” strings can be found by search engines, but if you are looking for foreign language sites, it might be still useful such as looking for “Webverzeichnis, Webkatalog” or the “Annuaire, Annu” words if you want to find German of French sites respectively.
You can add up some adjectives to these like “Free” or “SEO friendly” to the above expressions or other special keywords which can be found only on link directories (or sites dealing with link directories) such as for instance “ohne Backlinkpflicht” (“without compulsory backlink” in german) or otherwise narrow down to the type of directories you are looking for such as “deep link directory” or directories with “instant approval”.
The topic of the website you are about to promote with directory submissions can be also incorporated in these search queries, although a query like “Real estate directory” will scrape both directories accepting only real estate related links and subcategories of general directories.
Finding directories with footprints of directory scripts
Taking the most popular link directory script as an example, one expression which can be found on a huge number of link directories poweded by PHPLD happens to be the expression “powered by php link directory”, although there are many oher directories, where the owner has paid an extra fee to remove this attribute from the directory’s footer.
Other typical thing is the submission page’s URL: PHP Link Directory Script’s link submission script can be found at submission.php, so if you are looking for this sites with the inurl:submit.php special expression, you will find many PHP LD sites. Unfortunately many other systems or sites which have nothing to see with link directories have similar urls, therefore it is better ro refine this expression and use the inurl:”submit.php?c=” expression in Google, as the submission pages corresponding to certain categories have URLs like this.
Submission pages sometimes tend to have other specific texts, so examining a few directories of a certain kind you can figure out commonly used expressions (which are part of a default template of that directory script for instanca) you can find similar submission sites easily.
Anyway you can use Google and search for directory footprints expression to find a lot of footprint texts and expressions.
Finding sites by detecting links to submission forms
Although the words which are used by linking the submission forms show a great variety, you can discover which are the most commonly used expressions such as Add URL, Suggest a site, etc. The easiest is way of gathering all possible variations is to collect the verbs used for submission, suggestion and adding plus the words used for links/sites:
[enregister|proposer|proposez|s'inscrire|inscrire|referencer|réferencer|soumettre|ajouter|ajoutez|présenter] [votre|un|mon|une] [site|site internet|site web|lien|url]
As you can see in the above example we gathered different verbs and their most commonly used forms, then added other possible variations for the most requently used nouns in this context. Having put them in the word spinning syntax used by Allsubmitter’s value generation module and let processed by this module, you will get a long list of every possible combination:
“Enregister votre site”
“Enregister votre site internet”
“Enregister votre site web”
“Enregister votre lien”
“Enregister un site”
“Enregister un site internet”
“Enregister un site web”
“Enregister un lien”
“Enregister site”
“Enregister site internet”
“Enregister site web”
“Enregister lien”
“Proposer votre site”
“Proposer votre site internet”
“Proposer votre site web”
“Proposer votre lien”
“Proposer un site”
“Proposer un site internet”
“Proposer un site web”
“Proposer un lien”
“Proposer site”
“Proposer site internet”
“Proposer site web”
“Proposer lien”
and so on….
Some of them will be grammatically incorrect or for other reasons seldom used, but this will not be a problem, because if no site has these bad expressions normally created by machine translation only then it will not even give you false positives.
If you cannot find enough directories
Before you start to systematically discover link directories with a specialised tool like Allsubmitter, you can usie a directory list published on some web site as a starting point: although many directory lists are rather outdated, these will help you to identify what’s common in certain types of directories, thus give you a good start for creating your own expressions to scrape search engines in your search of directories.
As search engines will provide you with only a limited number of results for each keywords and expressions you might want to try expressions narrowed down to certain top level domain names, so instead of “Link Directory”, you can use the
- “link directory” site: com
- “link directory” site: net
- “link directory” site: org
- “link directory” site: info
- etc. expressions too
This way link directories will give you four times more results hopefully, as having only a few hundred initial results is just not enough given on the fact that how many of them will be filtered later on during the process.
If you’ve found just too many sites
Using the above methods you will have a whole lot of false positives: sites which are not directories, sites — like this directory blog — which only write about directory scripts or submissions. Allsubmitter has a few handy features to filter these lists and eliminate unnecessary items:
First of all you can determine which expressions’ presence would indicate in the directory that a site has a submission form and let Allsubmitter to find exactly where the submission forms are located too. Cases when sites which have internal links with texts usually referring to submission forms but on the referred subpage no actual forms can be found by the Allsubmitter (because for instance the form is only available for logged-in users) can be also determined.
Unfortunately this algo is not working 100% perfectly so you will still have many sites which will not be directories indeed, therefore you should also use an other feature of Allsubmitters’ Catalogue Base handling module which lets you filter lists by the contents of database fields. For instance if you search for %link% of %Link% or %dire% or %Dire% or similar SQL expressions in URLS and web page Titles but you cannot found neither the link nor the directory keywords, chances are high that you can remove that site from the scraping’s results.
Going through all results manually
Once you have systematically gathered and meticulously filtered your results, the last step would be to go through those sites with Semi-Automatic Submission to check whether they are really directories, and whether they allow your sites to be suggested for free or with any other conditions which are acceptable for you.
Allsubmitter provides sophisticated tools for the whole process, effectively assisting you to:
- gather sites with search engine queries
- remove duplicated links or domains from the results
- detect submission forms by examining web sites while deleting all sites which are not responding or giving errors
- determine other parameters necessary when assessing the results such as Page rank values or IP addresses
- finalize your directory list by going through the directories with an easy semi-automatic link submission process.
Search terms used previously:
- inurl:/users/index php username real name registration date last login promulgated
- deseado inurl:/join php als mitglied kannst du
- admirables inurl:/join php as a member you will be able to
- powered by phpld submit php
- inurl:/users/index php username real name registration date last login msci
- spinoza inurl:/join php as a member you will be able to
- ahead inurl:/guestbook php?action=
- scampoli inurl:/index php?action=register
- stead inurl:/index php?t=
- hiesset inurl:/index php?site=forum_topic
Related Posts
Tags
Share This
