Christopher Webber: Moving "Clarity Project" off of teamspace onto wiki.cc.org at mike's request.

2011-04-25T20:50:44Z

Moving "Clarity Project" off of teamspace onto wiki.cc.org at mike's request.

New page

Not too long ago we worked on the "Sanity Overhaul", a rewriting of
the CC website engine and underlying components (cc.i18n, cc.license,
cc.licenserdf, etc). While we all seem to agree that it's been a
massive improvement, it feels like there are a lot of loose ends to
tie up with the project. This page's purpose is to document what
those things are, and how to improve them.

We've made things (a lot more) sane. Now it's time to make them a lot more clear.

== cc.license ==

=== Exceptions suckiness ===

Exceptions suck in cc.license. Everything throws CCLicenseError, no
matter what the kind of error really is. It's impossible to tell what
specific problems have happened.

These exceptions are also abused, thrown when a query returns no
results in various parts of the infrastructure and caught later, where
returning None would have worked just as well or better.

'''Proposed solutions:'''
* Subclass CCLicenseError with more specific errors
* Use None where more appropriate than throwing an exception

=== by_code, by_url, etc monstrosities ===

There are quite a few "by_code" "by_url" and etc functions in
cc.license and they are all convoluted and mind-melting to observe.
The cult of the flying spagetti monster may be great, but not in
regards to our code. We should refactor these to make sense, and many
less tables will be flipped in the process.

=== shouldn't have to pass in model to rdf_helper calls ===

If rdf_helper functions are designed for specific models, shouldn't we
just call those models? Or at least, pass them in as keyword
arguments with default values, such as model=JURI_MODEL. That would
make a lot more sense, assuming we wanted to be able to do dependency
injection for unit testing purposes or something (which we don't
currently do anyway!). Or, you know, we could just call JURI_MODEL in
the code and save ourselves an argument.

=== cc.license._lib contains stuff which is useful to external programs ===

The namespacing of _lib implies that this stuff should be internal but
all sorts of things (okay, cc.engine at least) just imports stuff out
of it.

=== RDFa tests for Public Domain Mark, CC0 formatters ===

http://code.creativecommons.org/issues/issue679

We have string tests which are fragile (kind of useful in its own
way). We should have RDFa tests though.

== cc.engine ==

=== Not enough, not good enough unit tests ===

cc.engine doesn't have the best of coverage currently. We should fix that.

See also
[http://code.creativecommons.org/issues/issue732 Roundup: Add a string test to every page in cc.engine]

=== request.urlgen useless with /ccengine-fcgi/ ===

See [http://code.creativecommons.org/issues/issue669 Issue669]

=== wsgi_cache sometimes writes 0 byte files instead of caching ===

http://code.creativecommons.org/issues/issue698

A good candidate for [http://dustycloud.org/tmp/tableflip.txt table flipping].

I've wondered if we could at least apply a better bandaid so that 0
size cache files aren't written by:

* ''RewriteCond TestString -s'' in apache, which makes sure the file is > 0 bytes
* Upgrading wsgi_cache to see if the file == 0 bytes, if so, overwrite

Confirmation, Nathan thinks this is the way to go.

=== Deployment documentation not good ===

There's kind of some documentation on [[Jurisdiction Management]] but
it's not very good, and that's not really the right place.

Instead we should make clear documentation on
[[cc.engine deployment]] and instead just link to it from
[[Jurisdiction Management]].

=== A script that updates cc.engine for us ===

Now that we have the webadmin user, there's no reason not to just run
something like this on live:

$ ccengine_update

Which does:
* a git pull of cc.engine
* buildout
* reload of apache

And warns if there are any errors.

Maybe with the addition of this argument:

$ ccengine_update --clear-cache

It will also wipe the cache directory.

Also,

$ ccengine_update --clear-cache-only

== license.rdf ==

=== Jurisdiction management doc needs cleanup ===

[[Jurisdiction Management]] mostly has correct information now.

* verify that the info is right
* remove the red/blue/green cruft.

=== cc.licenserdf.tools.* all need tests ===

Some of these things have tests, not much though.

=== Consistent ordering in rdf writing ===

Currently whenever we write out RDF files, they change completely in
their ordering of assertions. This makes git diffs impossible, and
with files as large as index.rdf, bloats our repository unnecessarily
all the time. Even just sorting the assertions somehow alphabetically
would be wonderful.

== cc.i18n ==

=== Document historical reasonings of our .po file formats ===

We have, for example:
* '''cc/i18n/master/cc_org.po:''' The master .po file
* '''cc/i18n/po/es/cc_org.po:''' The transifex-edited translations for es
* '''cc/i18n/i18n/es/cc_org.po:''' The converted (to po-style? cc-style?? I always forget) files for compiling to .mo
* '''cc/i18n/mo/es/cc_org.po:''' The compiled .mo gettext files

How did we end up with our "unusual" gettext file setup? Why are
things this way? I don't even really know these things for sure and I
even rewrote some of the tools that work with them!

=== Do we still need master/cc_org.po ===

nkinkade thinks maybe not, since transifex just works with
po/en/cc_org.po anyway. We should figure this out.

=== Tests for our tools! ===

Can we haz?

== General ==

=== Convert everything to use cc.i18n.util.locale_to_lower_[lower|upper]() ===

http://code.creativecommons.org/issues/issue787

Clarity Project - Revision history

Christopher Webber: Moving "Clarity Project" off of teamspace onto wiki.cc.org at mike's request.