Dreamwidth Development

AI and Dreamwidth

Wed, 25 Feb 2026 20:21:52 GMT

Posted by: mark

We've seen some questions lately about AI and how it relates to Dreamwidth, especially around scraping and training. Rather than answer piecemeal, I wanted to talk through how denise and I are thinking about this and try to be explicit about some things.

Dreamwidth is a user-supported service. We don't build the service around monetizing user data, and that informs how we approach AI just like it informs everything else we do.

Your content and AI training

Dreamwidth does not and will not sell, license, or otherwise provide user content for AI training. We have not and will not enter into data-access agreements for AI training purposes.

We will continue taking reasonable technical steps to discourage large-scale automated scraping, including known AI crawlers, where it is practical to do so. No public website can prevent scraping with absolute certainty, but we will keep doing what we reasonably can on our side.

AI features on Dreamwidth

Dreamwidth will not introduce AI features (and we have no current intention of doing so) that use or process user content without a public discussion with the community first.

We're only phrasing it like this because we can't predict the future and who knows what will be possible and available in five or ten years, but right now there's nothing we can see wanting to add.

If that ever changed, the conversation would happen openly before any decisions were made.

Site admin uses of AI

Keeping Dreamwidth usable means dealing with things like spam and abuse, and that sometimes requires automated admin tools to be more efficient or effective.

We are not currently using AI-driven systems for moderation or similar decisions.

If we ever decide that an AI-based tool would help address a site admin problem like spam, we will explain what we are doing and how it works (and ask for feedback!) before putting it into use. Any such tools would exist only to make it easier and more efficient for us to do the work of running the site.

AI and code contributions

Dreamwidth is an open-source project, and contributors use a variety of tools and workflows.

Contributors may choose whether or not to use AI-assisted tools when writing or reviewing code. Dreamwidth will not require contributors to use AI tools, and we will not reject contributions solely because AI-assisted tools were used.

For developers: if you use any AI-assisted development tools for generating a pull request or code contribution, we expect you to thoroughly and carefully review the output of those tools before including them in a pull request. We would ask the community not to submit pull requests from automated agents with no human intervention in the submission process.

I think it's important and I want to be able to review, understand, and maintain any contributions effectively, and that means humans are involved and making sure we're writing code for humans to work with, even if AI was involved.

Important note: this applies to code only. We expect any submitted images or artwork (such as for styles, mood themes, or anything else) to be the work of a human artist.

And to be very explicit, any AI-assisted development does not involve access to Dreamwidth posts or personal content.

In short summary

Dreamwidth does not and will not provide user content for AI training
Dreamwidth have not and will not enter data-sharing agreements for AI training and we will do what we can to prevent/discourage automated scraping by AI companies
Dreamwidth will not introduce AI features without a public discussion first
Any site admin use of AI tools will be explained openly and part of a public conversation
Contributors can choose their own development tools for code, but we do not accept images or artwork generated by AI

Oh, and we'll probably mention this (or a subset of this that isn't code related) in an upcoming dw_news post, but will defer to denise on that!

comments

Overdue Announcement: Dreamwidth, Github and you

Sun, 26 Aug 2012 07:26:19 GMT

Posted by: foxfirefey

As of several weeks ago, all commits to the Dreamwidth codebases have gone to our new repositories on Github:

There are a few relevant wiki documents that have been fully revised to account for this change:

Moving your Dreamwidth installation to use Github -- these instructions will tell you how to move your current Dreamhack/dev environment over to a Github based installation
Dev Maintenance -- this document tells you how to keep your Github Dreamwidth based installation (and your Dreamwidth forks) updated with the code from the Dreamwidth repositories
Draft: Github development process -- This is the document in the least refined state, so keep in mind that it is in stronger need of suggestions and revisions. It goes through the very basics of Git workflow. This and Dev Maintenance might eventually end up merged into one document.

We are working on getting the rest of the wiki development documentation updated (see the dw_wiki post). Feel free to comment to this post with your questions/concerns about the move!

comments

Important: Perl 5.10 now considered required for DW installs

Wed, 18 Jul 2012 05:15:43 GMT

Posted by: mark

Hi all,

We're here at OSCON (O'Reilly's Open Source Conference in Portland) with a bunch of the volunteers who work on Dreamwidth. Related to this, we've finally decided to force an upgrade to our Perl base version required.

kareila says it well:

Hello all! A few facts have recently come to our attention:

1) Perl 5.10 has a significant number of new, desirable features over Perl 5.8.

2) Perl 5.10 was released 5 years ago, so it's reasonable to assume most people are using Perl 5.10 or newer in production.

3) We are, in fact, using Perl 5.10 in production.

Therefore, we are planning to start requiring Perl 5.10 as we fix bugs and add new features in the future. If you are running a Dreamwidth installation on a version of Perl older than 5.10, now would be an excellent time to consider upgrading.

We are also considering moving to 5.12 or 5.14 soon. We need to do some more investigation and also validate the system on those versions. But for now, consider 5.10 as required.

Thanks!

comments

update-db and the -r flag

Fri, 15 Jun 2012 03:44:06 GMT

Posted by: mark

Hi all,

This is important. Please note!

If you are going to be touching bin/upgrading/update-db-general.pl (and related scripts) -- you must respect the way this script works -- i.e., this script should never make any changes unless the -r flag is provided or it prompts the user running it.

This script is designed to be run by admins before code pushes, and it's supposed to spit out the "this is what I'm going to do" information. That way the admin can run it to see what might happen before it happens. It's not safe to just have it execute SQL without that flag.

This is a safety issue. I just ran it on production and it made some changes (regarding fixing certain edges) without warning me or requiring me to use the run flag. That's scary. :)

Thanks!

comments

jQuerify journals status

Wed, 21 Mar 2012 22:48:29 GMT

Posted by: foxfirefey

Bug 3532, the jQuerify journals bug, is moving towards completion. On the next code push, there will be a post to dw_beta for any remaining issues people are having. They'll be fixed and/or filed, depending on urgency, so that we can then turn on the jQuery for everyone, while still giving people a chance to turn the jQuery off. I imagine once all the issues are taken care of, the old journal Javascript will be deprecated and then removed.

For development, this means patches for old journal Javascript will no longer be accepted. If you see a bug open for old journal JS, it can be closed. (Don't do that if it's something that involves doing JS for the new JQuery version, though.)

( Extra info on: Why JQuery-izing? )

comments

LJ::EventLogRecord and LJ::EventLogSink: remove completely or finish implementation?

Fri, 23 Sep 2011 14:20:37 GMT

Posted by: pauamma

So Dreamwidth has an incomplete implementation of the LJ::EventLogRecord and LJ::EventLogSink system. Specifically, it's missing LJ::EventLogSink and LJ::EventLogSink::*, and thus doesn't do anything now except waste CPU time and RAM (assuming it's even used anywhere - it's disabled in the stock etc/config*.pl and in the Dreamwidth production configuration). (Note: this has no connection with the ESN system.)

If completed (mostly by importing the missing bits from LiveJournal), it would let us:
- selectively log to the database events listed in http://code.livejournal.org/trac/livejournal/browser/trunk/cgi-bin/LJ/EventLogRecord/
- feed some or all of the same events to an indexer (for full text search of journals) in a way similar to http://code.livejournal.org/trac/livejournal/browser/trunk/cgi-bin/LJ/EventLogSink/SearchIndexer.pm

So it has potential uses if completed, but since Dreamwidth has its own indexer/search engine and no one called for completing it, I think we may as well remove it. Does anyone see any reason for completing it instead? Completing it would mean reverting part of bug 164 - the database logging would still stay out, but the feed-to-indexer bits would return, probably in a modified form).

What do y'all think?

comments

Versions of perl

Sun, 15 May 2011 16:07:05 GMT

Posted by: pauamma

So Perl 5.14 was released today, and Perl 5.10 (and Perl 5.8) is no longer maintained. This raises the question: which version(s) of Perl should we allow, recommend, or use in development and production?

comments

New JS implementation on Journals // jQuery Code Guidelines

Tue, 29 Mar 2011 09:38:37 GMT

Posted by: fu

We just released a beta-test of the jQuery implementation on journals. You can turn it on by going to the Dreamwidth Beta page.

You can test even if you're not a dev-type, but please don't spread this link around too widely just yet. There's still a lot left unimplemented (most notably quick reply), and until the most widely-used user-facing bits are done, it's going to seem pretty broken.

(I'd rather not have random people who don't see the warning in this entry come away with the impression that we all just broke / are planning to break Dreamwidth).

When things are ready to go public, we will be setting up a public beta procedure, and post in the appropriate locations. For now, please if you go turn on beta, expect only comment moderation and deletion to work :)

You'll probably want to save the link so you can go turn it off at will.

For developers, I've put up on the wiki some instructions for putting future features in beta.

There's a lot of ways to set up code in jQuery; here are some things that have been working for me. Putting up for discussion; I'd like to start setting up guidelines to make it easier to get started quickly, soon.
( thoughts on standardizing jQuery implementation )

comments

Bugzilla etiquette, responsibilities, and business priorities

Thu, 01 Apr 2010 23:42:36 GMT

Posted by: mark

Recently we had a bit of confusion in Bugzilla about exactly who does what, who's responsible for what, and when it might be appropriate to circumvent someone else's priority in order to get something done more quickly. (Read: step in on someone else's bug to fix something.)

The situation: let's say that denise writes a patch. Then afuna comes along and commits that patch. A week later, I push it live, woo! It turns out, oh noes!, there was a bug in that code that went live. Who is responsible for fixing this?

The short and correct answer is: once code has gone live on the site and contains a bug, it is everybody's responsibility to ensure a timely fix is made.

When it comes down to it, this is a balancing act between Dreamwidth the community project and Dreamwidth the business. We spend most of our time optimizing for the former because I believe that's the right thing to do. However, from time to time, we have to make concessions in the name of the latter. (Example, I'd much rather be working on some features right now, but I need to do the payment system. Also, The review queue needs some serious work, but alas, I have to deprioritize it for a little while!)

This balancing act comes to the front again when we talk about the difference between code that is still theoretical (in Bugzilla as patches) and code that is live and available for users to interact with. As soon as code is live, I expect anybody who can step in to fix problems to do so, and I expect people who submit patches to understand that once it goes live, they lose a lot of that ownership that they are given while working on it to begin with.

I don't want people being afraid to fix bugs because of code/bug ownership or interpersonal issues -- we're all on the same team here. We're all working together to make sure this site continues to be as amazing as it has been, to keep a wonderful home for us and our friends, etc etc. This is something that is going to take people working together to accomplish, and 'this is my bug, go away' can be counter-productive to those goals in some cases.

Now, also consider that it does depend somewhat on the severity of the bug in question. If it's a minor issue (colors/font wrong, text out of place, ugly, etc) then you should ask the original author first before you touch the bug and start fixing it. It's just polite. In this example, Denise put all her time into the original feature or fix, more than likely she feels bad that it was broken and would like the opportunity to fix it. If Afuna swoops in and fixes it, that hinders Denise's ability to learn and it doesn't let her fully resolve the issue. (If in doubt on priority, ask Denise or me -- we'll be happy to tell you.)

But, and I want to stress this again, the decision on priority is going to be one that is fuzzy. Denise may think the bug is not worth an immediate fix from someone else, but Afuna might think it is. If Afuna comes in and fixes it, then Denise might be annoyed, but Afuna did nothing wrong. If the priority was wrongly determined, feel free to point that out and discuss it, but let's avoid any anger and frustration: everybody here did the right thing and was working to the betterment of the site.

Does anybody have any thoughts on this subject? I'm happy to discuss it and try to find something that elegantly balances the priorities and needs of the project with the goals and desires of our development community.

comments

Committers: crediting commits

Thu, 09 Jul 2009 23:17:58 GMT

Posted by: mark

Just like we attribute each commit to the person who submitted it, if it's a codemerge from LJ or some other site, it is required that we annotate that in the commit.

"Patch from LiveJournal."

That sort of thing. Please remember to do this, and if you haven't done it anywhere, it'd be great if you could go back and edit the changelog posts to include this note.

Thanks!

comments