In the last weeks Lemmy has seen a lot of growth, with thousands of new users. To welcome them we are holding this AMA to answer questions from the community. You can ask about the beginnings of Lemmy, how we see the future of Lemmy, our long-term goals, what makes Lemmy different from Reddit, about internet and social media in general, as well as personal questions.

We’d also like to hear your overall feedback on Lemmy: What are its greatest strengths and weaknesses? How would you improve it? What’s something you wish it had? What can our community do to ensure that we keep pulling users away from US tech companies, and into the fediverse?

Lemmy and Reddit may look similar at first glance, but there is a major difference. While Reddit is a corporation with thousands of employees and billionaire investors, Lemmy is nothing but an open source project run by volunteers. It was started in 2019 by @dessalines and @nutomic, turning into a fulltime job since 2020. For our income we are dependent on your donations, so please contribute if you can. We’d like to be able to add more full-time contributors to our co-op.

We will start answering questions from tomorrow (Wednesday). Besides @dessalines and @nutomic, other Lemmy contributors may also chime in to answer questions:

Here are our previous AMAs for those interested.

  • Lena@gregtech.euEnglish
    0·
    9 days ago

    Do you have any plans to make it easier to manage the images stored in pictrs? One issue I have is that I used to proxy images, I no longer do that, but now I have like 300GB on backblaze doing nothing. In this post I outlined more precisely what I mean.

  • Jeena@piefed.jeena.netEnglish
    0·
    10 days ago

    I think the greatest strength is that it is so compatible with other Threadyverse software like PieFed and Mbin. This brings a lot of freedom to the users.

    • nutomic@lemmy.mlOPMEnglish
      0·
      9 days ago

      Yes this is a major benefit of an open network. Lemmy is a very large project already, so it takes a lot of effort to implement new features, because they have to meet high standards for quality and performance and also work together with all the existing features. A project like Piefed is much smaller and can implement new features more quickly. This allows for more experimentation, and successful features can later be added to Lemmy.

      Also users who are not happy with Lemmy for any reason can switch to a different platform while still interacting with those on Lemmy. So if Piefed and Mbin grow that is also a benefit for Lemmy.

      • illpillow@lemmy.mlEnglish
        0·
        10 days ago

        Absolutely agree on this point. My first app didn’t fulfill my requirements, so I just tested another one being able to configure it how I like.

    • Dessalines@lemmy.mlMEnglish
      0·
      9 days ago

      Yes I’m very excited about the growth of other fediverse software, and a lot of the cool new features they’re adding. Its a great eco-system where we can experiment, be creative, and learn from each other.

  • Blaze (he/him) @lemmy.dbzer0.comEnglish
    0·
    10 days ago

    Hello,

    Thank you for organizing this AMA!

    Starting with a quite expected question: when do you think you’ll be able to release Lemmy 1.0?

    • Dessalines@lemmy.mlMEnglish
      0·
      9 days ago

      With the rate ppl are adding issues (and we’re finding more), is sometimes feels like it keeps getting farther away than nearer, but we’ll get there in some months.

    • nutomic@lemmy.mlOPMEnglish
      0·
      9 days ago

      Its hard to say because these things always take longer than expected. Now we are finally getting to the point where all the breaking database and api changes are almost finished. After that it will take some months to update lemmy-ui for all the backend changes and new features, and the same for all other apps. Then a testing period to fix all the problems that come up. So maybe around autumn for the final release, although lemmy.ml and some other instances may upgrade some months before already.

  • Blaze (he/him) @lemmy.dbzer0.comEnglish
    0·
    10 days ago

    What’s something you wish it had? What can our community do to ensure that we keep pulling users away from US tech companies, and into the fediverse?

    One of the biggest issue at this point is probably the registration experience. There are quite a few occurrences on [email protected] of users not sure whether their email has been validated or not, and at the moment they really need to look out for the toastify notification on their first try, later attempts won’t show it.

    Most recent example: https://lemmy.ml/post/27607055?scrollToComments=true

    If there could be a way to inform a user saying “your email address has been validated, please wait for an administrator to activate your account, you can reach out to them at xxx”, that would be great.

  • land@lemmy.mlEnglish
    0·
    10 days ago

    Native push notifications would be awesome for Lemmy! I’m keeping my fingers crossed.

  • sabreW4K3@lazysoci.alEnglish
    0·
    10 days ago

    Are you disappointed with the way things are growing with people trying to marginalise the likes of ML and Grad?

    • Sleepless One@lemmy.mlEnglish
      0·
      9 days ago

      I get a chuckle out of the “Tankie Triad” talking point some people keep using. It sounds like a villain organization from a Saturday morning cartoon.

    • nutomic@lemmy.mlOPMEnglish
      0·
      9 days ago

      It seems some people simply need some target to hate on. Hopefully they will learn to accept different opinions when they arent being manipulated by for-profit social media anymore.

    • Dessalines@lemmy.mlMEnglish
      0·
      9 days ago

      The anti-communist witch-hunters are extremely peeved that they can’t remove our communities like they can on reddit. Overall it doesn’t bother me because I don’t work for them, and they can always go back to reddit where their views are already dominant.

      Anyone trying to make the world a better place, will always be hated and hunted by some people; it’s a fact of life, and the sooner we accept it, the better.

    • KimBongUn420@lemmy.mlEnglish
      0·
      10 days ago

      Communities that go against hegemonic capitalist/imperialist discourse will always get marginalised. Not being able to take down those communities easily like on Reddit is a huge win by itself for Lemmy. The software offers a valuable savehaven for e.g ex r/chapotraphouse, r/genzedong etc.

      • Cowbee [he/they]@lemmy.mlEnglish
        0·
        10 days ago

        Yep, the fact that Communists can build their own platform and networks free from any outside censorship on corporatized platforms is itself the strategy for building leftist spaces. The goal isn’t hurt by more non-Communists being on the overall Lemmy platform because these non-Communists can’t actually do much to shut the Communists out.

  • totallyNotARedditor@lemm.eeEnglish
    0·
    10 days ago

    We are seeing an influx of new users, but what’s happening to older users? Are they still active? What’s the average lifetime of Lemmy users nowadays? I’m kinda curious about the user retention in general

    • nutomic@lemmy.mlOPMEnglish
      0·
      9 days ago

      I believe they are still active. User numbers have been stable for a long time, and there are some names that I recognize from the very early days 5 years ago.

    • Dessalines@lemmy.mlMEnglish
      0·
      9 days ago

      Every server and community has monthly active users stats. Best way to see them would be a tool like this that keeps track of history: https://lemmy.fediverse.observer/stats

      We don’t do any tracking of user retention, but overall lemmy has been fairly steady at ~50k users for a year now.

    • comfy@lemmy.mlEnglish
      0·
      10 days ago

      but what’s happening to older users? Are they still active?

      There are certainly names still around who I remember from my first year on the site. Like Blaze said, I’m also not sure how to get some concrete numbers.

    • mstrk@lemmy.worldEnglish
      0·
      8 days ago

      I’ve been here for almost two years and don’t think I can go back to anything else. I like the freedom of information that this idea brought to us.

  • poVoq@slrpnk.netEnglish
    0·
    10 days ago

    What are your thoughts on blocking AI scraper access? Any attempts to improve that on the side of Lemmy? Basic things like allowing to customize the robots.txt easily would already help.

    I also recently tried this new AI block tool called Anubis with Lemmy, but for some reason it fails with Lemmy-ui. Might be interesting to investigate further.

    • nutomic@lemmy.mlOPMEnglish
      0·
      9 days ago

      You can load a different robots.txt in your nginx config, something like this:

      location /robotx.txt {
    index /path/to/my/robots.txt;
}

      Additionally 1.0 will change the “private instance” to work with federation enabled (see https://github.com/LemmyNet/lemmy/pull/5530). Then only logged-in users will see content, while AI scrapers wont see anything except the login page.

    • Skelectus@suppo.fiEnglish
      0·
      10 days ago

      I just set up Anubis today. Specifically I’m only testing it for Lemmy-ui, and it seems to work fine.

      It looks like the distributed waves that keep bringing the service down hit exclusively our lemmy-ui subdomain, so maybe non-SSR photon is also a good defense, heh.

      • poVoq@slrpnk.netEnglish
        0·
        10 days ago

        Hmm, that is odd. I guess I need to double check my Nginx config for lemmy-ui then. You have your setup documented somewhere?

        Edit: ah, you run Photon as the main UI and lemmy-ui somewhere else? I think specifically the split between frontend and backend on the root domain somehow makes Anubis fail to set the correct cookie.

        • Skelectus@suppo.fiEnglish
          0·
          10 days ago

          I don’t think it should be a problem, but I’m not that sure either. Lemmy.fedi.zutto.fi also runs it and that’s just a normal lemmy-ui installation. I think Zutto simply forwarded all traffic to Anubis and then fixed federation. There was some discussion and config shared in sopuli’s finnish matrix room.

    • OsrsNeedsF2P@lemmy.mlEnglish
      0·
      10 days ago

      I’ve previously worked in anti-scraping. There is a negative 0% chance the Lemmy devs have the resources to effectively do this without tanking the server for everyone else.

    • Vent@lemm.eeEnglish
      0·
      10 days ago

      Anyone that wants to scrape Lemmy would have an easier time setting up their own server, federating with everyone, and reading straight from their DB. No web scraping required. Though, web scraping defenses would be useful against general web scrapers/crawlers.

      • poVoq@slrpnk.netEnglish
        0·
        10 days ago

        That would require the authors of these AI scrapers to actually give a f*ck. The problem is that they don’t, and just scrape what ever they can find repeatatly almost like a ddos attack on the open web.

  • seang96@spgrn.comEnglish
    0·
    10 days ago

    On the server perspective, I have a question, what are your thoughts for horizontal scaling on the database? This seems to be the biggest limitation and requiring higher spec hardware to scale especially for the bigger instances.

    My tiny instance for example I give over 20GB of RAM just to postgres to make it perform efficient enough.

      • seang96@spgrn.comEnglish
        0·
        9 days ago

        Yeah I used pgtune as a base and found more memory needed to be assigned to certain spots especially to keep federation with bigger instances, otherwise timeouts would occur resulting in my instance being constantly behind.

        That said I read postgres 17 is much more memory efficient, though I have yet to move my lemmy database to it yet since its the largest haha.

        • nutomic@lemmy.mlOPMEnglish
          0·
          9 days ago

          Maybe your disk is too slow, or latency between Lemmy and Postgres is too high?

          • seang96@spgrn.comEnglish
            0·
            9 days ago

            It is a k8s cluster and using ceph for all of my storage so the latency from that I bet is the largest reason and upping the memory offsets the disk writes. i also have another postgres DB syncing as a fallback for high availability. Fortunately after tuning the database and giving it enough RAM my instance has been running pretty stable for over a year without any changes.

            I am also using less powerful computers for the entire infrastructure (not server grade) which brings to the point of having horizontal scaling on database I imagine will be a growing need with growing instances, communities, and users since it can be cheaper to run multiple smaller spec servers rather than a single with the added benefit of high availability.

            • nutomic@lemmy.mlOPMEnglish
              0·
              9 days ago

              Postgres supports sharding which should work without any changes in Lemmy. But so far not even lemmy.world needs that. There are also read replicas which would require support directly in Lemmy afaik. Such a feature will surely be added as instances grow bigger over time and need more resources.

              • seang96@spgrn.comEnglish
                0·
                9 days ago

                I didn’t think of using read only replicas, that would probably be a very good way to go since its probably 80%+ of actions are reads. Thanks for answering, I am excited to see the how lemmy grows and thanks for all the devs hard work!

    • Dessalines@lemmy.mlMEnglish
      0·
      9 days ago

      The way to solve the database problems isn’t to keep throwing more and more money at powerful servers and scaling. Its to fix it at the root: lemmy’s unoptimized database.

      @dullbananas has done invaluable work in making our DB better (and all of these will be in 1.0), but I’m convinced that if we had even 1-2 more Postgresql experts do a pass over the DB, and ideally one full-time expert, all of these problems could be solved.

        • Dessalines@lemmy.mlMEnglish
          0·
          9 days ago

          We’ve asked for help various times, but don’t usually get much help. Despite the seemingly large number of “experts” out there, only a tiny number of them contribute to open source. I’d still consider it mostly a wasteland, with a few people doing the work that should be done by 100x their number.

      • seang96@spgrn.comEnglish
        0·
        9 days ago

        I 100% agree with this and there have been great strides since I started using Lemmy ~v0.17! That said at some point optimization will have lower returns and have a higher effort to put into and once a community grows extensively it likely might not be enough, so I was curious to what you guys were thinking at that point, something like Ctius for sharding postgres?

        • Dessalines@lemmy.mlMEnglish
          0·
          9 days ago

          I’m sure we’re nowhere near that level yet. We haven’t come close to postgres’s limits, and most of our bottlenecks are unoptimized queries.

        • Dessalines@lemmy.mlMEnglish
          0·
          9 days ago

          The post list query is by far the worst offender. It needs to filter, sort, cursor paginate, and join to many tables, and indexes are hard to follow and keep up with.

          What’s more is that the problems only surface with lots of historical data, meaning we can only really test the query plans with a fully populated DB.

          All this requires running lemmy locally, and inspecting the postgres query durations. We really need proper test suites (lemmy DB perf is one example) that can stress-test production data also.

          Here is one historical issue:

          I’d very much appreciate any help.

          • subversive_dev@lemmy.mlEnglish
            0·
            9 days ago

            Good evening Dessalines, I have started looking at the posts query.

            The lowest hanging fruit I think would be if we could replace some of the joins with WHERE EXISTS which can have a huge impact on the query time. It seems this is supported in Diesel: https://stackoverflow.com/a/74300447

            This is my first time looking at the codebase so I can’t tell yet which joins are purely for filtering (in which case they can be replaced by WHERE EXISTS) and which joins need to be left in because some of their columns end up in the final SELECT

            I can’t tell for sure yet but it also looks like this might also be using LIMIT...OFFSET pagination? That can be a real drag on performance but isn’t as easy to fix.

            EDIT:

            Looking some more, and reading some linked github discussion - I think to really get this out of the performance pits will require some denormalization like a materialized view or manual cache tables populated by triggers. I really like the ranking algorithm but so far I’m finding it difficult to optimize from a query perspective

  • MemmingenFan923@feddit.orgEnglish
    0·
    10 days ago

    Some companies use Reddit as their main forum or an established way to communicate with customers. Are there any companies that have explored Lemmy and have their community yet?

  • abobla@lemm.eeEnglish
    0·
    10 days ago

    Do you guys have plans to add a spoiler tag? I post a lot of memes about tv shows that I watch, but the users complain that the post isn’t blurred.

    I know I can use the NSFW tag, but this gives the wrong idea and limits the post visibility (since people can hide nsfw posts).

        • Zagorath@aussie.zoneEnglish
          0·
          10 days ago

          Yeah a lot of former Reddit apps that switched to Lemmy did a really lazy job of it and haven’t implemented all of the Lemmy text parsing syntax properly. Spoilers are one of the most common issues, but so are subscript (including ~multiple word subscript~) and superscript (and ^multiple words of it^).

          If your app doesn’t parse text correctly 2 years later, it may be time to consider switching.

      • abobla@lemm.eeEnglish
        0·
        10 days ago

        but then the user might not realize that there’s an image in the post, which will also limit it’s reach.

          • Dessalines@lemmy.mlMEnglish
            0·
            9 days ago

            Even that isn’t too necessary, since you can already put images in spoiler markdown blocks.

          • abobla@lemm.eeEnglish
            0·
            9 days ago

            I see. Would the spoiler tag also blur the thumbnail?

            The only thing that concerns me about handling spoilers is how the third party apps handle them. Do you think it would be a good idea to also blur the entire image (not only the thumbnail) and remove the blur only when the user clicks the image?

            • nutomic@lemmy.mlOPMEnglish
              0·
              9 days ago

              Not sure, we would have to see whenever we get around to implementing that.