karzilla: a green fist above the word SMASH! (Default)
[staff profile] karzilla
We're about to deploy a new backend interface for file storage, called BlobStore, which [staff profile] mark wrote over the past few months with the intention of standardizing how file storage is handled in our code and making it work with any number of possible underlying technologies. It currently supports MogileFS and local disk, and we plan to add support for S3 in the future.

At this point, MogileFS is considered legacy technology. If your site is set up to use MogileFS, that configuration will continue to work under BlobStore for now. However, no new code that requires MogileFS will be accepted.

What you need to know if you are writing code: the new methods are implemented in cgi-bin/DW/BlobStore.pm and are pretty straightforward. For the most part they serve as drop-in replacements for the MogileFS file methods.

What you need to know if you are running a server: if you try to do anything related to uploading images, including userpics, you will get a fatal error unless you have defined either %LJ::BLOBSTORE or %LJ::MOGILEFS_CONFIG. So if you were already using MogileFS, you're fine, but if not, you will need to set up local disk storage in one of your local config files. The stock etc/config-private.pl in dw-free will have an example %LJ::BLOBSTORE that you can uncomment and use.

What you need to know if your existing userpics disappear: If you were running a server without MogileFS, all of your system's userpics were stored in a database table, and use of that table is no longer supported. I'm working on a new version of the migrate-userpics.pl script that can be used to move the images into your BlobStore once you've got that configured. (Update: this is available in bin/upgrading/migrate-userpics.pl.)

Obviously this will all need to be documented on the wiki somewhere, but I've got my hands full right now making sure everything is nailed down to push these changes into production in a few days. Let me know if there's anything I didn't cover here that needs to be addressed.
kareila: IT prepares you for a life of fighting with PCs nonstop. (sysadmin)
[personal profile] kareila
http://wiki.dwscoalition.org/wiki/index.php/MogileFS_setup

All of the information that was previously there assumed you had root on a Debian box. I've updated the page to document the steps needed to get it working for dreamhacks as well. (Until the next time things break, anyway.)

YOU'RE WELCOME.

Ahem.

That is to say, if you try using my instructions and run into problems, let me know!
fu: Close-up of Fu, bringing a scoop of water to her mouth (Default)
[personal profile] fu
Hmm, it looks like there's been a lot of changes going on in the mogile repository right now -- after running an update, I get a warning that I can't start mogilefsd because I'm on the wrong database schema.

Heads-up in case anyone runs into the same thing -- I've rolled back to r1459 of mogile for now.

Profile

dw_dev: The word "develop" using the Swirly D logo.  (Default)
Dreamwidth Open Source Development

March 2017

S M T W T F S
   1234
5678 91011
12131415161718
1920212223 2425
262728293031 

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Mar. 26th, 2017 05:21 am
Powered by Dreamwidth Studios