{"id":22081,"date":"2022-09-28T12:30:23","date_gmt":"2022-09-28T12:30:23","guid":{"rendered":"https:\/\/www.openbusinesscouncil.org\/?p=22081"},"modified":"2022-09-28T12:30:23","modified_gmt":"2022-09-28T12:30:23","slug":"public-data-collection-is-advancing-but-still-far-from-its-full-potential","status":"publish","type":"post","link":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/","title":{"rendered":"Public Data Collection Is Advancing, But Still Far From Its Full Potential"},"content":{"rendered":"

The web scraping industry is maturing both from the technology and business perspective, however, it still lacks proper regulation. For this reason, key market players are launching an Ethical Web Data Collection Initiative (EWDCI) to share best practices and advocate for common principles. These were some of the main takeaways from this year\u2019s edition of the prominent industry conference \u2014\u00a0OxyCon<\/a>.<\/strong><\/p>\n

\"Public
Public Data Collection Is Advancing, But Still Far From Its Full Potential<\/figcaption><\/figure>\n

Organized by a leading public web data gathering solutions provider Oxylabs<\/a>, OxyCon connected global web scraping experts for a two-day online event. From practical tips for engineers to high-level panel discussions, the conference speakers reviewed the most recent developments in the field.<\/p>\n

Allen O’Neill,\u00a0\u00a0CEO and CTO at\u00a0The DataWorks<\/a>, argued that while the web scraping industry has been developing rapidly over the years, there\u2019s still so much potential left for the future:<\/p>\n

\u201cThe web scraping industry hasn\u2019t even scratched the surface with its potential yet. There will be many new unicorns in the industry in the upcoming ten years – those who will be able to harness the power of information extraction (not data extraction, but information extraction) and use that to gain insights that have never been seen before\u201d,<\/em> – said Allen.<\/p>\n

The fast growth of the industry was illustrated by scaling being the hottest topic at OxyCon. Karsten Madsen, CEO at SEO company\u00a0Morningscore<\/a>, shared the story of his team moving from small data requests to having to compete with SEO industry giants. According to him, it\u2019s not always about having the most data or the smartest data – it\u2019s about having smarter algorithms to manage it.<\/p>\n

Glen De Cauwsemaecker, Lead Crawler Engineer at\u00a0OTA Insight<\/a>\u00a0had another tip for scaling data operations: \u201cBe pragmatic and look for cost-reward balance\u201d,<\/em> – he recommended to the fast-growing data companies.<\/p>\n

Besides the technical challenges of scaling, legal issues are also often close to the top of the list of concerns. The participants of the panel discussion \u201cLawyers discuss scraping\u201d<\/em> emphasized the ambiguity and many unclear areas that come with the lack of proper industry regulation. As a result, the industry itself must be proactive in safeguarding it from within and sharing best practices among each other.<\/p>\n

In this light, Christian Dawson, Executive Director at\u00a0I2Coalition<\/a>\u00a0made an announcement of a new web scraping industry initiative. I2Coalition, together with 5 public data aggregators – Oxylabs, Zyte, Smartproxy, Coresignal, and Sprious has launched an\u00a0Ethical Web Data Collection Initiative (EWDCI)<\/a>. The aim of the group will be to promote the industry\u2019s best practices and advocate for beneficial technical standards.<\/p>\n","protected":false},"excerpt":{"rendered":"

The web scraping industry is maturing both from the technology and business perspective, however, it still lacks proper regulation. For this reason, key market players are launching an Ethical Web Data Collection Initiative (EWDCI) to share best practices and advocate for common principles. These were some of the main takeaways from this year\u2019s edition of […]<\/p>\n","protected":false},"author":7,"featured_media":22083,"comment_status":"closed","ping_status":"closed","sticky":true,"template":"","format":"standard","meta":{"_mo_disable_npp":""},"categories":[25,3822,3501],"tags":[22148,22146,22149,22147,22145],"acf":[],"yoast_head":"\nPublic Data Collection Is Advancing, But Still Far From Its Full Potential - OpenBusinessCouncil Directory<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Public Data Collection Is Advancing, But Still Far From Its Full Potential\" \/>\n<meta property=\"og:description\" content=\"The web scraping industry is maturing both from the technology and business perspective, however, it still lacks proper regulation. For this reason, key market players are launching an Ethical Web Data Collection Initiative (EWDCI) to share best practices and advocate for common principles. These were some of the main takeaways from this year\u2019s edition of […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\" \/>\n<meta property=\"og:site_name\" content=\"OpenBusinessCouncil Directory\" \/>\n<meta property=\"article:published_time\" content=\"2022-09-28T12:30:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.footballthink.com\/wp-content\/uploads\/2022\/09\/Public-Data-Collection-Is-Advancing-But-Still-Far-From-Its-Full-Potential.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Hernaldo Turrillo\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Hernaldo Turrillo\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\"},\"author\":{\"name\":\"Hernaldo Turrillo\",\"@id\":\"https:\/\/www.footballthink.com\/#\/schema\/person\/b9610afd0759dc701187a7f622375c23\"},\"headline\":\"Public Data Collection Is Advancing, But Still Far From Its Full Potential\",\"datePublished\":\"2022-09-28T12:30:23+00:00\",\"dateModified\":\"2022-09-28T12:30:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\"},\"wordCount\":456,\"publisher\":{\"@id\":\"https:\/\/www.footballthink.com\/#organization\"},\"keywords\":[\"conference\",\"OxyCon\",\"OxyCon conference\",\"Oxylab\",\"Public Data Collection Is Advancing But Still Far From Its Full Potential\"],\"articleSection\":[\"News\",\"Resources\",\"Technology\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\",\"url\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\",\"name\":\"Public Data Collection Is Advancing, But Still Far From Its Full Potential - OpenBusinessCouncil Directory\",\"isPartOf\":{\"@id\":\"https:\/\/www.footballthink.com\/#website\"},\"datePublished\":\"2022-09-28T12:30:23+00:00\",\"dateModified\":\"2022-09-28T12:30:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.footballthink.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Public Data Collection Is Advancing, But Still Far From Its Full Potential\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.footballthink.com\/#website\",\"url\":\"https:\/\/www.footballthink.com\/\",\"name\":\"OpenBusinessCouncil Directory\",\"description\":\"Openbusinesscouncil\",\"publisher\":{\"@id\":\"https:\/\/www.footballthink.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.footballthink.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Organization\",\"Place\"],\"@id\":\"https:\/\/www.footballthink.com\/#organization\",\"name\":\"openbusinesscounsil\",\"url\":\"https:\/\/www.footballthink.com\/\",\"sameAs\":[],\"logo\":{\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo\"},\"image\":{\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo\"},\"openingHoursSpecification\":[{\"@type\":\"OpeningHoursSpecification\",\"dayOfWeek\":[\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\",\"Sunday\"],\"opens\":\"09:00\",\"closes\":\"17:00\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.footballthink.com\/#\/schema\/person\/b9610afd0759dc701187a7f622375c23\",\"name\":\"Hernaldo Turrillo\",\"description\":\"Hernaldo Turrillo is a writer and author specialised in innovation, AI, DLT, SMEs, trading, investing and new trends in technology and business. He has been working for ztudium group since 2017. He is the editor of openbusinesscouncil.org, tradersdna.com, hedgethink.com, and writes regularly for intelligenthq.com, socialmediacouncil.eu. Hernaldo was born in Spain and finally settled in London, United Kingdom, after a few years of personal growth. Hernaldo finished his Journalism bachelor degree in the University of Seville, Spain, and began working as reporter in the newspaper, Europa Sur, writing about Politics and Society. He also worked as community manager and marketing advisor in Los Barrios, Spain. Innovation, technology, politics and economy are his main interests, with special focus on new trends and ethical projects. He enjoys finding himself getting lost in words, explaining what he understands from the world and helping others. Besides a journalist, he is also a thinker and proactive in digital transformation strategies. Knowledge and ideas have no limits.\",\"url\":\"https:\/\/www.footballthink.com\/author\/hturrillo\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo\",\"url\":\"https:\/\/www.footballthink.com\/wp-content\/uploads\/2017\/04\/logo_big.png\",\"contentUrl\":\"https:\/\/www.footballthink.com\/wp-content\/uploads\/2017\/04\/logo_big.png\",\"width\":1161,\"height\":250,\"caption\":\"openbusinesscounsil\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Public Data Collection Is Advancing, But Still Far From Its Full Potential - OpenBusinessCouncil Directory","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/","og_locale":"en_US","og_type":"article","og_title":"Public Data Collection Is Advancing, But Still Far From Its Full Potential","og_description":"The web scraping industry is maturing both from the technology and business perspective, however, it still lacks proper regulation. For this reason, key market players are launching an Ethical Web Data Collection Initiative (EWDCI) to share best practices and advocate for common principles. These were some of the main takeaways from this year\u2019s edition of […]","og_url":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/","og_site_name":"OpenBusinessCouncil Directory","article_published_time":"2022-09-28T12:30:23+00:00","og_image":[{"width":1200,"height":675,"url":"https:\/\/www.footballthink.com\/wp-content\/uploads\/2022\/09\/Public-Data-Collection-Is-Advancing-But-Still-Far-From-Its-Full-Potential.jpg","type":"image\/jpeg"}],"author":"Hernaldo Turrillo","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Hernaldo Turrillo","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#article","isPartOf":{"@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/"},"author":{"name":"Hernaldo Turrillo","@id":"https:\/\/www.footballthink.com\/#\/schema\/person\/b9610afd0759dc701187a7f622375c23"},"headline":"Public Data Collection Is Advancing, But Still Far From Its Full Potential","datePublished":"2022-09-28T12:30:23+00:00","dateModified":"2022-09-28T12:30:23+00:00","mainEntityOfPage":{"@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/"},"wordCount":456,"publisher":{"@id":"https:\/\/www.footballthink.com\/#organization"},"keywords":["conference","OxyCon","OxyCon conference","Oxylab","Public Data Collection Is Advancing But Still Far From Its Full Potential"],"articleSection":["News","Resources","Technology"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/","url":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/","name":"Public Data Collection Is Advancing, But Still Far From Its Full Potential - OpenBusinessCouncil Directory","isPartOf":{"@id":"https:\/\/www.footballthink.com\/#website"},"datePublished":"2022-09-28T12:30:23+00:00","dateModified":"2022-09-28T12:30:23+00:00","breadcrumb":{"@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.footballthink.com\/"},{"@type":"ListItem","position":2,"name":"Public Data Collection Is Advancing, But Still Far From Its Full Potential"}]},{"@type":"WebSite","@id":"https:\/\/www.footballthink.com\/#website","url":"https:\/\/www.footballthink.com\/","name":"OpenBusinessCouncil Directory","description":"Openbusinesscouncil","publisher":{"@id":"https:\/\/www.footballthink.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.footballthink.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Organization","Place"],"@id":"https:\/\/www.footballthink.com\/#organization","name":"openbusinesscounsil","url":"https:\/\/www.footballthink.com\/","sameAs":[],"logo":{"@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo"},"image":{"@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo"},"openingHoursSpecification":[{"@type":"OpeningHoursSpecification","dayOfWeek":["Monday","Tuesday","Wednesday","Thursday","Friday","Saturday","Sunday"],"opens":"09:00","closes":"17:00"}]},{"@type":"Person","@id":"https:\/\/www.footballthink.com\/#\/schema\/person\/b9610afd0759dc701187a7f622375c23","name":"Hernaldo Turrillo","description":"Hernaldo Turrillo is a writer and author specialised in innovation, AI, DLT, SMEs, trading, investing and new trends in technology and business. He has been working for ztudium group since 2017. He is the editor of openbusinesscouncil.org, tradersdna.com, hedgethink.com, and writes regularly for intelligenthq.com, socialmediacouncil.eu. Hernaldo was born in Spain and finally settled in London, United Kingdom, after a few years of personal growth. Hernaldo finished his Journalism bachelor degree in the University of Seville, Spain, and began working as reporter in the newspaper, Europa Sur, writing about Politics and Society. He also worked as community manager and marketing advisor in Los Barrios, Spain. Innovation, technology, politics and economy are his main interests, with special focus on new trends and ethical projects. He enjoys finding himself getting lost in words, explaining what he understands from the world and helping others. Besides a journalist, he is also a thinker and proactive in digital transformation strategies. Knowledge and ideas have no limits.","url":"https:\/\/www.footballthink.com\/author\/hturrillo\/"},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.footballthink.com\/public-data-collection-is-advancing-but-still-far-from-its-full-potential\/#local-main-organization-logo","url":"https:\/\/www.footballthink.com\/wp-content\/uploads\/2017\/04\/logo_big.png","contentUrl":"https:\/\/www.footballthink.com\/wp-content\/uploads\/2017\/04\/logo_big.png","width":1161,"height":250,"caption":"openbusinesscounsil"}]}},"_links":{"self":[{"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/posts\/22081"}],"collection":[{"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/comments?post=22081"}],"version-history":[{"count":1,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/posts\/22081\/revisions"}],"predecessor-version":[{"id":22084,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/posts\/22081\/revisions\/22084"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/media\/22083"}],"wp:attachment":[{"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/media?parent=22081"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/categories?post=22081"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.footballthink.com\/wp-json\/wp\/v2\/tags?post=22081"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}