momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
[personal profile] momijizukamori
I have been researching this, and while I've found a lot of articles on how to set up faceted searching using existing search engines (mostly Solr, though a few with Sphinx), and a bunch of articles on frontend design for faceted search.... there is not a lot on optimizing data structure or queries that I can find. And I know basically nothing about code optimization so - tossing this out here!

Data structures )

Query Structure )

Tragically the already existing options are 'use Solr', 'use something written on a Java server backend', or 'use a client-side Javascript library (which won't work with JS off and will probably not work on a 1300+ collection, but we didn't stress-test it to see)'
ninetydegrees: Drawing: chibi version of me (chibi)
[personal profile] ninetydegrees
Here are the new locations:

bin/upgrading/s2layers/... => styles/...
bin/upgrading/s2layers.dat => styles/s2layers.dat

ext/dw-nonfree/bin/upgrading/s2layers/... => ext/dw-nonfree/styles
ext/dw-nonfree/bin/upgrading/s2layers.dat => ext/dw-nonfree/styles/s2layers.dat

If you're working on a styles bug, you may want to save your files, update your code and work on a new branch (or do what's necessary to update your current branch and reapply your changes).
momijizukamori: Grey tabby cat with paws on keyboard and mouse. The text reads 'code cat is on the job', lolcats-style (CODE CAT)
[personal profile] momijizukamori
(Apologies to anyone seeing this twice, I posted it in [site community profile] dw_dev_training but Mark and D suggested here was probably a better fit!)

So, this is the gigantic project that's been kicking around my head the last few months, because I love our themes but right now, the organization is nooooot there. And I love organization. The full bugzilla listing has all the debates and links and what not, but I'm going to add the summary of UI implementation I ran by Denise below:

quoted from Bugzilla )

Because the scope of this runs through Perl, MySQL, BML, and Javascript, today I sat down and wrote out a vague work-flow for how looking at themes by category in /customize should run, and how adding themes as an admin should run, so I have an idea of what I need to do.

very rough, mind you )

I'd really appreciate feedback on if I missed any important stuff, logistics of the UI implementation, and how exactly we should store this information in the databases. Or even on the list of categories I have so far, though there will probably be a bigger, more official RFC when things get far enough.
ninetydegrees: Drawing: girl peeking through blinds (peeking)
[personal profile] ninetydegrees
Dear fellow devs,

bug 4726 will move all folders in /bin/upgrading/s2layers/ as well as /bin/upgrading/s2layers.dat to /bin/styles. I have this mostly patched so, if you're currently working on any of these files, please set your bug as blocking mine so I can keep my patch up-to-date and know when it's safe to push it.

Thank you!

P.S. baggyeyes, I've already done this for your bug as I know you're working on it. :)
ninetydegrees: Drawing: girl peeking through blinds (peeking)
[personal profile] ninetydegrees
If you make any change to Core2 --frontend but also backend, please check whether this also affects any of our official styles and open a bug for them if you don't want to update them as part of your own bug. It's the only way to make sure everything stays up-to-date and functional, and only requires a simple 'grep -ril'. :)
[personal profile] swaldman
I've been working on Bug 1723, and while doing that I've noticed something that I don't think is right.

The following information on tag usage is exposed to the style system (via the TagDetail class):
  1. use_count: The number of times that the tag in question is used in the journal (at present it's the total number of times, but bug 1723 will most likely change it to being an approximation of the number of times that it's used in posts that the current user can see. This isn't important now :-))
  2. visibility: The most-relaxed security level that the tag is ever used in.
  3. security_counts: How many times it is used at each security level, including security levels that the current user does not have access to.
(1) is fine, and is used by all styles (there is a privacy leak at present, but bug 1723 is meant to fix that)
(2) seems like info that styles don't really need to know, but it can't hurt
(3) concerns me, because it exposes information that the current user shouldn't have.

For example, let's say that user A has not been granted access by user B. User B posts a lot with the tag "hatemyboss". Most of these posts are protected or private, but one of the posts is public. Because of the public post, user A is able to see the "hatemyboss" tag. If user A were to go away and write his own style that used the info in (3) above, he would be able to find out how often user B has used "hatemyboss" in protected, private and filtered posts.

I'm told that none of DW's official styles use the info in (3). This doesn't mean that a custom style doesn't use it somewhere, on DW or another site using the code.

I think that there are three options:
(a) Leave things as they are
(b) Stop exposing the security_counts variable to the styles system
(c) Keep the variable, but force all the values to zero

I prefer (c), because it deals with the privacy issue without totally breaking any custom styles that use this information. We could add a note in core2.s2 explaining what is going on, and maybe remove the variable totally in a year's time or something.

What are other peoples' opinions?

If anybody wants to have a look for themselves, search for:
  • "class TagDetail" in core2.s2
  • "$t->{security_counts}" in
momijizukamori: Green icon with white text - 'I do believe in phosphorylation! I do!' with a string of DNA basepairs on the bottom (Default)
[personal profile] momijizukamori
...mostly because as anyone who gets all the bugzilla and/or style notifs knows, I did a lot of that this week *g*

First, preview images - I'd been doing them with Ubuntu's screenshot utility and photoshop, which is 1) really slow and 2) doesn't really adequately strip and compress images. So! Enter Screen Capture for Google Chrome, which I've been using with the 'full page' setting, and set to skip pop-ups and save straight to a temp image directory, where I can then rename the screenshots. After that, [personal profile] geekosaur and [personal profile] exor674 (with a little sideline ImageMagick input from [profile] tamouse_ helped me write this handy little shell script, to process all the pngs in the directory you run them on - it resizes them, crops them, strips color profiles, and indexes them.

I've been using it on my own machine, as I run Ubuntu, but I believe all the appropriate packages are now on the Dreamhack server, too, for server-side use.

Second, [personal profile] rb posted a handy little script for sorting properties for color themes and generating appropriate headers. I updated it to match our current formatting for files, add sorting for a few more properties, and output more verbose instructions on what to do next at the end.

This is written to be used server-side right now, with a command-line input like './ "Human-Readable Theme Name" user "Layout This Theme Is For" yes < inputfile' I've been running it client-side by changing line 74, 'my $filename = "$ENV{LJHOME}/temp/$layout_name-$theme_name.s2";' to 'my $filename = "temp_themes/$layout_name-$theme_name.s2";' (otherwise it tries to write to root, and That's Bad. And unnecessary)

Third, for the tiny handful of people doing S2 work on gedit (shhh, I like it, okay), I made some mods to the CSS syntax highlighting file (pastebin because the file is big). I tried to get it to make a custom S2 language, but it was being fussy about parsing it, and a lot of the .s2 files are largely CSS anyway. So this adds highlighting for S2 variables and S2 keywords! It goes in a file called CSS.lang in home/.local/usr/share/gtksourceview-2.0/language-specs, which will probably not exist if you haven't made custom languages. If you're on newer versions that use GTK3, that 2.0 should be a 3.0. You may have to manually select the highlighting, for some reason it only registers the .s2 extension some of the time.
exor674: Computer Science is my girlfriend (Default)
[personal profile] exor674
I have noticed certain in-journal links such as next and previous do not preserve the style= setting and really should.

Can people please look at links and compile a list of places that do not preserve this option so I can open a bug.

jeshyr: Dreamwidth: Dream wide, dream deep (Dreamwidth - Dream wide Dream Deep)
[personal profile] jeshyr
Right before my brain got refried by this stupid disease I was working on a script to turn the theme layers copied from DW itself into the right format to paste into the s2 code in the sourcecode. I was doing it as I learned how so it's only been tested on about 5 different situations at the moment and I'm quite sure it doesn't do everything so check all output please!!

And yes, my brain got fried (again) by this stupid disability so I'm on indefinite leave from developing for the mean time - and VERY disappointed about it.

At the moment you want to pipe the theme layer through STDIN to the script, so something like

$ ./ NewThemeName Rising "Tabula Rasa" yes < ~/file-with-theme-layer.txt

it will tell you what file it writes the output to, it will be something in $LJHOME/temp/


Feedback, cases that break it, improvements, etc. would all be great.


Edit: Now with super bonus functioning HTML tags!


dw_dev: The word "develop" using the Swirly D logo.  (Default)
Dreamwidth Open Source Development

March 2017

5678 91011
1920212223 2425


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Mar. 26th, 2017 05:25 am
Powered by Dreamwidth Studios