Questions as opposed to problems this time

[whobutdrew]

Yeah... its me again. So much for not spamming the comm, eh? :P

My site is live now. I have a couple of questions surrounding the regular operation of it.

First, is there a way to disable the username.subdomain.tld functionality, so that the site only uses http://subdomain.tld/users/username? I ask this because my site is 'hosted' through Dyndns.com. It will support the resolution of *.subdomain.tld, but only if I pay them. Since I'm a miser I'd like to avoid that if possible. Manually typing in http://subdomain.tld/users/username will work, but all of the links use http://subdomain.tld/users/username, which fail. Its all of $15/year, which I can probably swing soon enough, but if there's a way to adapt the 'old' way of doing things, I'd like to give it a shot. Answers along the lines of "You're SOL" will be accepted."

Second, and again because I love free solutions (that's why I'm here in the first place, after all...) I'm using a GMail account as an SMTP server. Will I have to keep TheSchwartz's worker-manager running in order for comment notifications and the like to work?

In a similar vein, I'm trying to download posts, comments, userpics, etc. from my pre-existing account to my server. I'm following the steps here: http://wiki.dwscoalition.org/notes/Importer_Testing

My first attempt seems to be stuck, though, even after a server reboot. The logs say that account verification has already been requested, but the queue (as visible from the webpage itself) has "ImportStatus" for the verification status, and it never progresses from there. So I suppose the question here is, is there a way to clear the queue so I can try again fresh?

If there's a better place to ask these 'usage' rather than 'installation and development' questions, point me in that direction and I'll try to stay good on my promise not to spam anymore!

You have all been a huge help, I wouldn't have made it this far without you. Thanks so much!

Comments

[kraig]

Can't speak to the ability to break out the username.domain.name bit from the code (I imagine it's possible), but there's a reason for it - it's so that people can't steal others' cookies, which use a host.domain.name format for trust. So kraig.dreamwidth.org can't see whobutdrew.dreamwidth.org's cookies with some malicious code, but dreamwidth.org/kraig can write some malicious code to see whobutdrew's cookies, if this theoretical malicious kraig could trick poor whobutdrew into visiting a page that had it. So unless you can explicitly trust all your users, I'd hold off on your plan.

[denise]

Yeah, it's an option in the config -- in config.pl, there are these lines:


# Support URLs of the form http://username.yoursite.com/ ?
# If so, what's the part after "username." ?
$USER_VHOSTS = 1;
$USER_DOMAIN = $DOMAIN;

# If you ONLY want USER_VHOSTS to work and not the typical /users/USER and /community/USER
# then set this option:
$ONLY_USER_VHOSTS = 1;


That's what controls the user subdomains, but as kraig says, it's a security issue and you really, really shouldn't do it unless you absolutely have to.

[exor674]

And to add to this -- the current patch that makes dw-free work alone actually will not work if user_vhosts are off -- I'e not got around to fixing that yet, so...
Should be a trivial fix though.

[exor674]

Do you have TheSchwartz set up? The workers don't really care if its set up or not and just run anyway (but won't be able to do anything, they just won't mention it either)

See http://wiki.dwscoalition.org/notes/TheSchwartz_Setup

That page is kinda out of date, the first step needs to be "visit http://yoursite.org/tools/importer" (not use the command line tool)

Also, a status of 'ImportStatus' means the task was aborted, so either you tried to start a second import before your other one completed, or I think that could possibly happen because you don't have TheSchwartz setup.

Also step #2 is that you have to run bin/worker/import-scheduler after each step (it's a one-off thing)

Then run bin/worker/content-importer-verify --verbose, wait till it finishes, ctrl-c it
run bin/worker/import-scheduler again
run bin/worker/content-importer --verbose

and keep running bin/worker/import-scheduler in another window until your job finishes.

You will have to keep the worker-manager running -- and I'm not sure if the default config has everything you'll need (I'll look into that) -- but it won't actually use the GMail account as the SMTP server, it'll send the mail itself. There might be a way to use a different server.

[whobutdrew]

I went through the steps for setting up TheSchwartz, but when the first import attempt got botched, I aborted it and tried again, so that would explain the "ImportStatus." Running content-importer/import-scheduler hasn't seemed to help move things along, but that's assuming I am running them as you intended me to, which I very well may not be doing.

I'm doing content-importer-verify --verbose, then ctrl-c once it starts saying "found no jobs." Then I'm running import-scheduler (no args), then content-importer (not verify) again. As I mentioned, no change on the import screen.

So could I just kick off a /bin/worker-manager & in one window, then work in another and have it constantly running in the background, firing off emails and jobs and such whenever it is called upon?

[exor674]

Okay, on my devsite, I start an import from somewhere

/tools/importer shows:

Profile data Waiting for verification to finish created 2 seconds ago
Journal comments Waiting for entries to finish created 2 seconds ago
Journal entries Waiting for tags and groups to finish created 2 seconds ago
Custom security groups Waiting for verification to finish created 2 seconds ago
Friends Waiting for groups to finish created 2 seconds ago
Tags Waiting for verification to finish created 2 seconds ago
Icons Waiting for verification to finish created 2 seconds ago
Verifying username/password Ready to be entered into the queue created 2 seconds ago

dw@dw-dev:~$ ~/bin/worker/import-scheduler
dw@dw-dev:~$ ~/bin/worker/content-importer-verify --verbose
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Verify'
Working on DW::Worker::ContentImporter::LiveJournal::Verify ...
job completed
TheSchwartz::work_once found no jobs
ctrl-c

Changes to

Profile data Ready to be entered into the queue created 1 minute ago
Journal comments Waiting for entries to finish created 1 minute ago
Journal entries Waiting for tags and groups to finish created 1 minute ago
Custom security groups Ready to be entered into the queue created 1 minute ago
Friends Waiting for groups to finish created 1 minute ago
Tags Ready to be entered into the queue created 1 minute ago
Icons Ready to be entered into the queue created 1 minute ago
Verifying username/password Finished successfully as of 34 seconds ago created 1 minute ago

dw@dw-dev:~$ ~/bin/worker/import-scheduler
dw@dw-dev:~$ ~/bin/worker/content-importer --verbose
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Userpics'
Working on DW::Worker::ContentImporter::LiveJournal::Userpics ...
<redacted>
[0.0000s 1.8s] Local userpic import complete.
job completed
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::FriendGroups'
Working on DW::Worker::ContentImporter::LiveJournal::FriendGroups ...
job completed
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Bio'
Working on DW::Worker::ContentImporter::LiveJournal::Bio ...
job completed
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Tags'
Working on DW::Worker::ContentImporter::LiveJournal::Tags ...
job completed
TheSchwartz::work_once found no jobs
ctrl-c

Changes to

Profile data Finished successfully as of 57 seconds ago created 3 minutes ago
Journal comments Waiting for entries to finish created 3 minutes ago
Journal entries Ready to be entered into the queue created 3 minutes ago
Custom security groups Finished successfully as of 58 seconds ago created 3 minutes ago
Friends Ready to be entered into the queue created 3 minutes ago
Tags Finished successfully as of 56 seconds ago created 3 minutes ago
Icons Finished successfully as of 58 seconds ago created 3 minutes ago
Verifying username/password Finished successfully as of 2 minutes ago created 3 minutes ago

dw@dw-dev:~$ ~/bin/worker/import-scheduler
dw@dw-dev:~$ ~/bin/worker/content-importer --verbose
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Friends'
Working on DW::Worker::ContentImporter::LiveJournal::Friends ...
<redacted>
job completed
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Entries'
Working on DW::Worker::ContentImporter::LiveJournal::Entries ...
<redacted>
[0.0127s 2.1s] counted 7 entries, lastgrab is now 2009-01-11 22:04:33.
job completed exits on it's own
dw@dw-dev:~$ ~/bin/worker/import-scheduler
dw@dw-dev:~$ ~/bin/worker/content-importer --verbose
$ ~/bin/worker/content-importer --verbose
TheSchwartz::work_once got job of class 'DW::Worker::ContentImporter::LiveJournal::Comments'
Working on DW::Worker::ContentImporter::LiveJournal::Comments ...
<redacted>
job completed

Changes to

Profile data Finished successfully as of 4 minutes ago created 6 minutes ago
Journal comments Finished successfully as of 1 minute ago created 6 minutes ago
Journal entries Finished successfully as of 2 minutes ago created 6 minutes ago
Custom security groups Finished successfully as of 4 minutes ago created 6 minutes ago
Friends Finished successfully as of 2 minutes ago created 6 minutes ago
Tags Finished successfully as of 4 minutes ago created 6 minutes ago
Icons Finished successfully as of 4 minutes ago created 6 minutes ago
Verifying username/password Finished successfully as of 5 minutes ago created 6 minutes ago

[whobutdrew]

With the queue currently in "ImportStatus" mode from earlier, I do import-scheduler/content-importer-verify, and it comes back with "Found no jobs" right away. :(

[exor674]

Oh, sorry, you need to start a new import (from /tools/importer) and it needs to show something like mine shows -- it shouldn't abort again.

[whobutdrew]

THAT worked. Perfectly.

I love you guys. :D

[exor674]

Also, bin/worker-manager will background itself, so no reason to run it in a separate window.

[ame]

Hi! I don't know how to help you, but I am interested in helping to beta test your website. :) May I?

[whobutdrew]

Thanks for the offer! There are plenty of technical issues that I have to work out before releasing to a 'beta' level of functionality (Thankfully for everyone else, they are issues I know how to tackle!)

'Tis a kind offer, though, and I thank you again for it.

[sophie]

By the way, if you're interested in a totally free DNS service that supports dynamic updating, I can personally recommend http://everydns.net/ . I use them myself. You do need to know a bit more about DNS than with DynDNS, but not too much, and it's possible to download clients to update the dynamic part of your domains.

If you need help, let me know. :)

[whobutdrew]

I think I will need to take you up on this. I finally got my site live again with DynDNS's wildcard support enabled, and it didn't appear to work.

If it'll be easier for you, I'll PM you my email address so we can renew this DNS discussion outside of a dated support thread. This will give me time to flesh out my site again before fiddling with DNS. Its been a while since I've dealt with raw DNS records, so bear with me!

[sophie]

...aw crap, I just realised I never replied to this.

So, um. I expect you probably don't need the help any more given it's nearly one year on, but if you do, let me know and I'll gladly help you out. <3

[whobutdrew]

Hah, no worries.

I've probably gone through about 3 fresh starts since this thread was posted. I tried to patch my non standard port/username.journalsite.com issue (I totally had it working once) but have been made of patch fail. I've since just gone with journalsite.com/users, and its been working, so I'm content to leave it with that.

Thanks for the thought, though!

[denise]

Also, I forgot to say in my earlier comment -- don't worry about asking questions here! We're happy to help out, and the more people we have poking at the code and the install process, the better. :)