szabgab: (Default)
Gabor Szabo ([personal profile] szabgab) wrote in [site community profile] dw_dev2011-05-16 09:16 pm
Entry tags:

The "entities" of a Dreamwidth setup

Hi,

Seeing how complex the deployment of applications developed in various corporations I am doing a little research to see how open source application manage the complexity. For that I was looking for complex (in terms of deployment and development environment) applications to see how they manage the task and how they test their system. Dreamwidth seems like a good case so I jumped on the IRC channel where [personal profile] pauamma gave me a quick list of the entities. Then I was volunteered to raise the issue here to get further clarifications.

So here is the list I got:


  1. perlbal on the load balancer(s)

  2. webservers running Apache+modperl+DW webapp code

  3. database cluster for global data

  4. several (currently 3) database clusters for per-user data. Each cluster holds the same data for different users

  5. Gearman servers, to offload some web-synchronous compute-intensive tasks

  6. TheSchwartz server(s) for asynchronous or non-web tasks.

  7. MogileFS servers for blobby data not stored in a SQL database for various reasons.

  8. Postfix



I found the Production Notes to be also a good source for information but I'd appreciate to get further details if the list is not complete. What would be very interesting to me is to understand what are the responsibilities of the various entities in the system. What kind of data is kept in the various databases. Which entities can be duplicated if load requires it and how the jobs can be divided?

Post a comment in response:

If you don't have an account you can create one now.
No Subject Icon Selected
More info about formatting

If you are unable to use this captcha for any reason, please contact us by email at support@dreamwidth.org