sfmlab
patreon
Development roundup: Summer 2025
🕑 Added 2025-08-02 04:20:38 +0000 UTCWhen did we have our last development roundup? Oh wow. Over a year ago? Jeez. Well, let's share about a year of updates with you then!
(This post is mirrored from https://open3dlab.com/news/development-roundup-summer-2025/ )
The end of 2024 was marked by highs and lows. After a year filled with unfortunate database, connectivity and file upload related incidents, a lot of focus for the site was on resilience and reliability. About a year ago, I implemented measures for people to verify download checksums, which helped in detecting synchronization issues. This unfortunately diverted time from my previous goal of improving the search features of the site.
Search improvements
Search is the most important element of the site. I want to provide people with the 3D resources that they need for their creative projects. If they can't find what they need, or if search queries take too long, they get frustrated and may close the site.
If you don't remember, a search on the site used to take multiple seconds to perform a particular lookup. This was due to the system doing SQL database lookups directly. This is very accurate, but much slower when compared to systems optimized for very dynamic search. Improving the search has been a long-time goal for years, but it's proven challenging for various reasons, the main one being my capacity to build something better, and the necessary money to afford the hardware to run it. At the end of summer in 2024, I was finally able to make the necessary changes and implement a new search backend. The new search backend was extremely fast, often reducing the time spent on search queries from 2 seconds to 200 milliseconds.
However, it ended up not being without its problems. The new backend proved difficult to tame, and initially, I had misconfigured it. And I had built a new search frontend to go with it. Once deployed, this blew up in my face. Search was practically unusable, and it took several days to correct it. But after correcting, the results were much faster and more in-line with expectations.
Payment Processor Compliance
Then, near the end of November, I had gotten search to a mostly functional state. I was due for a vacation, and left for Japan. I was intending to focus on improving the search when I got back from holiday. Returning from Japan, it was on to the Chaos Communication Congress (38C3) in Hamburg, where I went to see various presentations and connect with people in the European hacker community.
While walking around the congress, I received an email from SubscribeStar. Less than two years before, I worked to get the Subscribestar Connection on-par with Patreon's, which helped foster a small but dedicated community of backers. Not being fully dependant on a single source (Patreon) would be useful in troubling times. But unfortunately, this email would be about the exact bullshit I was worried about. Compliance requirements. It appears these requirements don't necessarily come from Subscribestar itself, but more so from their payment partners like VISA and Mastercard. As long as the Open3DLab integration with Subscribestar was not compliant to their standards, payments would be withheld. This would wipe any of my intent to focus on site usability and search quality off the table, and consume the first three months of development work for 2025.
Improvements were made to the connection. The process of connecting accounts is more obvious, more information about the sync process is visible, and there are behind the scenes improvements to the handling of the different benefit tiers. And these don't just apply to subscribers through Subscribestar, but also for those subscribing through Patreon. You can find the changes and account linking under your account settings on this very website.
Other parts of the compliance process meant having resilient reporting functionality in place. I spent most of February on constructing a report system to handle different kinds of abuse requests. Logged-in users can use these systems to easily file reports through a simple web form. We now also have backend systems for logging reports and moderation actions taken on those reports. People without an account will be presented with a button that will prompt them to send an email. Please be aware that any copyright, paywall violation or IP theft related issues must always be processed through email for legal reasons.
Similarly, mods now have tools to more easily nuke content that doesn't comply with site policy. One example is user avatars, which can be deleted with a single click. And when a DMCA takedown is submitted by a rights holder and acted upon by a moderator, a deletion notice is automatically posted to the Discord for transparency.
Tags and Search
It was extremely frustrating to have the first quarter of the year be consumed by tasks other than the ones I had intended to work on initially. My goals of working on search had been completely sidetracked. But in late March, it was time to get into it.
One of the main hurdles for the site search is the quality of tags and the ability for users to reliably shape their experience. People have complained about specific types of uploads or the uploads of specific users ever since the first of the sites launched in 2014. I've always had a vision of how this could be solved - comprehensive tagging, and having people filter certain tags and users.
Luckily, I'm happy to announce that it is now possible to block users from your search results. If you are tired of seeing the same 3 names cluttering the frontpage over and over and over and over again, it is now possible to block these users, and you will no longer see their uploads.
Tag blocking however, relies on people tagging their content according to the tagging guidelines, which is apparently too much to ask. Uploaders fuck up the tags consistently, ignore the guidelines and shit up the tag list with typo's all over the place, which people then happily copy into subsequent uploads. There was an option to correct these issues, and merge these tags into canonical tags, but that doesn't prevent people from making the same mistakes again, and adding completely wrong tags. For this, I've implemented a Tag Alias feature, which automatically replaces the wrong tags upon submission. In addition, completely useless tags can be assigned a blocked status, and will be filtered away automatically. I then merged and created aliases for various common tagging mistakes, which should make search using tags more reliable.
While shaping the tag experience is still an ongoing work for the moderation team, the early results are promising. And so far, most of the initial work has been done to let people start blocking tags in their profile. Blocking specific tags will prevent those uploads from showing up on the frontpage, as well as on the "more-like-this" section. The only thing that is missing at this time, is a mechanism for verified users to police the tag system. And just very basic UX improvements for the search element. But that is a topic for another time.
A new CDN
There have been various improvements to the platform that the site runs on. The biggest one has been to the delivery of CSS, Javascript and thumbnail assets. The site uses a new CDN for these, with points of presence all over the world. It should significantly reduce load times of these assets for users outside of Europe; even those in Africa, Asia and Oceania. If the site feels more responsive, it's likely because of the new CDN and the new site search backend. I would like to hear from you if you've noticed the recent improvement, so please leave your feedback in our Discord.
Similarly, I've reduced and optimized the number of static files (CSS, JS) files that are necessary to serve the site. All things considered - big speed gains from all of these changes.
Changes you may not have noticed (but your brain did).
And there's many more small improvements that you may not have noticed. We added a new markdown editor with better, more accurate previewing and a new "spoiler" element that works for both inline and block-level spoilers. There's also this very news page you're looking at now! Collection views are now also paginated so they don't take forever to load when they contain 200+ entries. There's a better administrator view for keeping track of sync tasks. There's a one-way message system to provide uploaders with feedback on their uploads, especially when intervention is necessary. Projects now have download statistics for all attached files combined. Sorting options on the main search are now more sensible and include total project downloads as a sort option.
And we are now keeping privacy respecting site statistics that allow us to identify pageview trends on a site level and page level. It is important to me that these statistics are kept without personal identifiable information - we only go as far as tracking whether a page view is anonymous, a signed-in user or a bot. We do not link this information to user accounts. I personally have little interest in keeping tracking data on users, as being responsible for that amount of data would require additional protection to keep that data safe and thus also increase legal liability. The primary reason for this increased interest in pageviews and site usage is that I lack insight into any kind of visitor statistics. All previous pageview numbers were estimates based on momentary captures and raw bandwidth numbers from service providers. Identifying bots is of extra interest at this time, due to the increase of AI scrapers. And an added benefit would be that these numbers could help in securing better sponsorships or ad deals than the ones currently available, although there's no concrete plans to do so as yet.
Lately, I've been working on resolving various papercuts that have bothered new users to the site. I've implemented measures that should prevent new users from entering their email address as their username - a surprisingly common error. Existing users who have landed themselves in this situation now also have the option to resolve this - they get the one-time opportunity to change their username to something that's not an email address. If you signed up with an email address as your username, I encourage you to make use of this asap.
And the latest addition: we can pin comments now! Yay.
For the future
There are many things I want to improve in the foreseeable future. Search is still unfinished. Tags are still unfinished. But there are threats on the horizon that I'm unsure how to account for. Various countries are implementing age verification using ID cards which, in my opinion, is an unreasonable requirement to put on small-time platform owners such as myself. I'm convinced it can not be done in a way that respects my privacy, is secure and safe on the part of the operator, and won't break the bank if outsourced to a third party. The UK is just the first to implement this, and other EU countries are set to follow. And the US is also not far behind. It's a farce.
And in addition to that, credit card companies are having a meltdown over some porn games. Which is equally worrysome as someone who operates a site like Smutbase. I'm looking at pre-emptive measures I can take to prevent from being struck by Patreon's compliance team. More news on that soon. At this time, I hope for your patience in this matter.
For now, I want to thank you for reading this, and thank you for supporting the site.
