• Tangent5280@lemmy.world
    link
    fedilink
    arrow-up
    0
    ·
    13 days ago

    It would be trivial to remove duplicates. With a little bit of foresight they could just as easily avoid the duplicates in the first place.

    • Damage@feddit.it
      link
      fedilink
      arrow-up
      0
      ·
      13 days ago

      They could also avoid to re-crawl the whole internet every day, but here we are, so who knows.