Web data encompasses various information drawn from web sites and apps. Typically, this tracks the connections between web sites and their visitors, including the amount of time people spent on the sites, and what they did there.
Web data comes from internal and more external sources. Internal sources are trackers like cookies and website and app analytics. External sources include web scrapers, canvas fingerprinting, keyword search data for a geographical area, and more.
Most websites and software programs have website analytics programs already available for you. These measure number of site visitors, click-through rate from off-site ads or connected social media pages, and amount of time spent on the site. They also track information like user demographics, whenever the devices they use record that data.
Other typical attributes of this data include IP address or mobile device ID.
Companies typically use this data to measure their marketing campaigns and audience reach. However, they also use it to conduct market research and assess the health of a website or app.
Many web hosting services provide analytics already. However, to test the data quality yourself, just make sure your dataset is complete, accurate, relevant, and updated frequently. Further, if you use web scraping tools, make sure they do not overwhelm the websites you are scraping data from, as that may get your tool blocked.
You can also follow the example of Leadbook, which takes a random sample of their data every quarter and manually checks the information to be sure it meets minimum expectations of accuracy.
Leadbook: Our Data
Wiley Online Library: Web Data: The Original Big Data – Taming the Big Data Tidal Wave
“A year [before the outbreak of deadly riots related to food shortages], on January 12, 2010, a tech startup posted an article on its blog: “Yemen heading for disaster in 2010?” The author, “Ninja Shoes”, wrote: “Based on the information we’ve gathered, Yemen will likely experience food shortages and torrential floods in 2010. This combination of natural disasters, propensity for famine and malnutrition, and challenges with Islamic radicals and terrorists, make it a hot spot for conflict in the future.”
Wired: The news forecast: Can you predict the future by mining millions of web pages for data?