alex [they, il]@jlai.lu to Technology@midwest.socialEnglish · 7 months agoThe rise and fall of robots.txtwww.theverge.comexternal-linkmessage-square0fedilinkarrow-up10arrow-down10cross-posted to: technology@lemmy.world
arrow-up10arrow-down1external-linkThe rise and fall of robots.txtwww.theverge.comalex [they, il]@jlai.lu to Technology@midwest.socialEnglish · 7 months agomessage-square0fedilinkcross-posted to: technology@lemmy.world
