{"id":65,"date":"2016-05-09T01:26:37","date_gmt":"2016-05-09T01:26:37","guid":{"rendered":"http:\/\/portal.venidera.com\/kb\/?p=65"},"modified":"2018-07-26T13:39:51","modified_gmt":"2018-07-26T16:39:51","slug":"download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems","status":"publish","type":"post","link":"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/","title":{"rendered":"Download de arquivos com PyCurl do CDN CloudFlare"},"content":{"rendered":"<div class=\"fusion-fullwidth fullwidth-box fusion-builder-row-1 hundred-percent-fullwidth non-hundred-percent-height-scrolling\" style=\"--awb-border-radius-top-left:0px;--awb-border-radius-top-right:0px;--awb-border-radius-bottom-right:0px;--awb-border-radius-bottom-left:0px;--awb-overflow:visible;--awb-flex-wrap:wrap;\" ><div class=\"fusion-builder-row fusion-row\"><div class=\"fusion-layout-column fusion_builder_column fusion-builder-column-0 fusion_builder_column_1_1 1_1 fusion-one-full fusion-column-first fusion-column-last fusion-column-no-min-height\" style=\"--awb-bg-size:cover;--awb-margin-bottom:0px;\"><div class=\"fusion-column-wrapper fusion-flex-column-wrapper-legacy\">Ao codificar em Python um download de uma URL que o CloudFlare CDN faz cache \u00e9 necess\u00e1rio informar um &#8220;User Agent&#8221; e ter suporte \u00e0 SSLv3, caso contr\u00e1rio voc\u00ea obter\u00e1 um arquivo com o nome do que requisitou mas que \u00e9 um HTML com os dizeres:<\/p>\n<blockquote>\n<div class=\"cf-wrapper cf-header cf-error-overview\">\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_75 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<label for=\"ez-toc-cssicon-toggle-item-69eaf19d4a991\" class=\"ez-toc-cssicon-toggle-label\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/label><input type=\"checkbox\"  id=\"ez-toc-cssicon-toggle-item-69eaf19d4a991\"  aria-label=\"Alternar\" \/><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#Error_1010_Ray_ID_29xxxxxxxxxed_%E2%80%A2_2016-05-08_19_20_51_UTC\" >Error 1010 Ray ID: 29xxxxxxxxxed \u2022 2016-05-08 19:20:51 UTC<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#Access_denied\" >Access denied<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#What_happened\" >What happened?<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-1'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#Error_1010_Ray_ID_29xxxxxxxxxed_%E2%80%A2_2016-05-08_19_20_51_UTC-2\" >Error 1010 Ray ID: 29xxxxxxxxxed \u2022 2016-05-08 19:20:51 UTC<\/a><ul class='ez-toc-list-level-2' ><li class='ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#Access_denied-2\" >Access denied<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/portal.venidera.com\/kb\/2016\/05\/09\/download-files-from-cloudflare-cdn-using-pycurl-to-solve-permissions-problems\/#What_happened-2\" >What happened?<\/a><\/li><\/ul><\/li><\/ul><\/nav><\/div>\n<h1><span class=\"ez-toc-section\" id=\"Error_1010_Ray_ID_29xxxxxxxxxed_%E2%80%A2_2016-05-08_19_20_51_UTC\"><\/span><span class=\"cf-error-type\" data-translate=\"error\">Error<\/span> <span class=\"cf-error-code\">1010<\/span> <small class=\"heading-ray-id\">Ray ID: 29xxxxxxxxxed \u2022 2016-05-08 19:20:51 UTC<\/small><span class=\"ez-toc-section-end\"><\/span><\/h1>\n<h2 class=\"cf-subheadline\" data-translate=\"error_desc\"><span class=\"ez-toc-section\" id=\"Access_denied\"><\/span>Access denied<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<\/div>\n<div class=\"cf-section cf-wrapper\">\n<div class=\"cf-columns two\">\n<div class=\"cf-column\">\n<h2 data-translate=\"what_happened\"><span class=\"ez-toc-section\" id=\"What_happened\"><\/span>What happened?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The owner of this website (website.domain.org) has banned your access based on your browser&#8217;s signature (xxxxxxxxxxxxxxxx-ua47).<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"cf-error-footer cf-wrapper\"><span class=\"cf-footer-item\">CloudFlare Ray ID: <strong>2xxxxxxxxxxxxed<\/strong><\/span> <span class=\"cf-footer-separator\">\u2022<\/span> <span class=\"cf-footer-item\"><span data-translate=\"your_ip\">Your IP<\/span>: 1xx.20x.xx.xxx<\/span>\u00a0<span class=\"cf-footer-separator\">\u2022<\/span> <span class=\"cf-footer-item\"><span data-translate=\"performance_security_by\">Performance &amp; security by<\/span> CloudFlare<\/span><\/div>\n<\/blockquote>\n<p>Realizei testes com urllib, urllib2, urllib3 e requests (que usa urllib), com todos ocorreram situa\u00e7\u00f5es que ou envolveram instalar pacotes demais ou n\u00e3o havia bom suporte \u00e0 SSLv3.<\/p>\n<p>A solu\u00e7\u00e3o foi utilizar o PyCurl, que se voc\u00ea usa Tornado e consome alguma API com seu cliente HTTP ass\u00edncrono provavelmente j\u00e1 ter\u00e1 instalado no seu ambiente (ou virtualenv).<br \/>\nSegue o c\u00f3digo de exemplo:<\/p>\n[code language=&#8221;python&#8221;]\nimport pycurl<\/p>\n<p>with open(&#8216;testimage.jpg&#8217;, &#8216;wb&#8217;) as f:<br \/>\n    c = pycurl.Curl()<br \/>\n    c.setopt(pycurl.USERAGENT, &#8216;Mozilla\/5.0 (Windows; U; Windows NT 6.1; it; rv:1.9.2.3) Gecko\/20100401 Firefox\/3.6.3 (.NET CLR 3.5.30729)&#8217;)<br \/>\n    c.setopt(c.URL,&#8217;https:\/\/website-in-cloudflare-cdn.domain.extension\/imagex.jpg&#8217;)<br \/>\n    c.setopt(c.WRITEDATA, f)<br \/>\n    c.perform()<br \/>\n    c.close()[\/code]\n<p>Depois de resolver com o c\u00f3digo acima eu encontrei (<a href=\"http:\/\/meta.stackexchange.com\/questions\/261741\/cloudflare-error-1010-banned-access-based-on-your-browsers-signature\">aqui<\/a>) onde desativar a checagem de integridade de navegador no CloudFlare (\u00fatil para quem \u00e9 administrador do site):<br \/>\nSettings-&gt;CloudFlare Settings-&gt;Browser Integrity Check-&gt;Toggle Off.<div class=\"fusion-clearfix\"><\/div><\/div><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11,12],"tags":[],"class_list":["post-65","post","type-post","status-publish","format-standard","hentry","category-development","category-python"],"_links":{"self":[{"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/posts\/65","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/comments?post=65"}],"version-history":[{"count":12,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/posts\/65\/revisions"}],"predecessor-version":[{"id":234,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/posts\/65\/revisions\/234"}],"wp:attachment":[{"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/media?parent=65"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/categories?post=65"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/portal.venidera.com\/kb\/wp-json\/wp\/v2\/tags?post=65"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}