The main website of the BSC is failing every day around 3:00 AM for almost one hour, so it is not a very good target. Instead, google.com is used which should be more reliable. The same robots.txt path is fetched, as it is smaller than the main page. Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Description
Old jungle repository with big blobs
Languages
HTML
48.3%
Nix
36.4%
C++
6.3%
CSS
5.8%
JavaScript
1.5%
Other
1.6%