Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
aboutsummaryrefslogtreecommitdiff
path: root/backend/lib
Commit message (Collapse)AuthorAgeFilesLines
* sync: add gentoo-ci pkgcheck results syncronisationMichael Palimaka2017-07-301-1/+62
|
* models: add a model for gentoo-ci pkgcheck resultsMichael Palimaka2017-07-301-0/+14
|
* sync: fix a missing .items to iterate on dictGilles Dartiguelongue2017-01-231-1/+1
|
* sync: add version and keyword synchronizationGilles Dartiguelongue2017-01-221-4/+38
|
* sync: add detail points to sync_versionsGilles Dartiguelongue2017-01-221-0/+15
|
* sync: use dict facilities for key retrieval with a defaultGilles Dartiguelongue2017-01-221-20/+19
|
* sync: use dict facilities for key retrieval with a defaultGilles Dartiguelongue2017-01-221-1/+1
|
* sync: reduce unneeded conditional evaluationGilles Dartiguelongue2017-01-221-2/+2
| | | | tags cannot be evaluated to go though these branches after the first if so switch to elif.
* sync: replace assert with ValueError raiseGilles Dartiguelongue2017-01-221-4/+5
| | | | Simpler expression, probably here to stay.
* sync: define project keys default valuesGilles Dartiguelongue2017-01-221-31/+33
| | | | | Costs less than checking for it in each loop iteration and does no harm later to loop on empty lists.
* sync: sort imports according to PEP8Gilles Dartiguelongue2017-01-221-2/+5
|
* sync: Use dict comprehension in sync_categories as wellMart Raudsepp2017-01-221-3/+1
|
* sync: use ORM magics in sync_packagesGilles Dartiguelongue2017-01-221-1/+1
| | | | | ORM knows howto map objects to ids through relationships so skip the details and focus on the thing you want to do.
* sync: use assert for GLEP67 compliance checkGilles Dartiguelongue2017-01-221-4/+6
| | | | Should never be raised actually but who knows.
* sync: Fix pkg sync for packages that have a same named pkg in another categoryMart Raudsepp2017-01-221-7/+1
| | | | | | Also fixes an InstrumentedList issue due to change from the categories.packages relationship from dynamic loading to select in commit 8d90fa1009 having broken that earlier
* sync: use dict-comprehension in sync_packagesGilles Dartiguelongue2017-01-221-3/+3
|
* sync: fix broken sync_packagesGilles Dartiguelongue2017-01-221-1/+6
| | | | | I think there is a problem in the logic here but at least this gets me past the initial sync.
* models: Add preliminary model and fields for keyword and p.mask storageMart Raudsepp2016-12-071-2/+22
|
* sync: Always handle e-mails in lower case to not end up with duplicatesMart Raudsepp2016-12-072-10/+15
| | | | Suggested-by: Doug Freed <dwfreed@mtu.edu>
* sync: Increase the sync delta to 1 hour and print the sync count and oldest ↵Mart Raudsepp2016-12-071-2/+5
| | | | TS at start
* sync: Don't forget to commit db transaction after all packages are syncedMart Raudsepp2016-12-071-0/+2
| | | | | Sometimes don't need to cancel out, so save the updates after the last modulo 100 to DB too :)
* sync: Order package details syncing based on how old the last sync wasMart Raudsepp2016-12-071-1/+1
| | | | | | This way if we got stuck and re-run much later (or it has exceeded the buffer time constant), we'll at least sync the oldest ones first, so we always end up being less out of date with the oldest sync ts.
* sync: Add package description and maintainers syncMart Raudsepp2016-12-071-4/+45
| | | | | | | | Maintains a sync timestamp to skip recently synced packages, so if a previous run got stuck, we can skip re-doing it too soon. Saves the DB transaction after every 100 packages, because packages.g.o seems to rate-limit us, so at least we will have things saved into DB periodically to cancel out when we get stuck and restart.
* models: Add package maintainers relationship table and ORM relationshipsMart Raudsepp2016-12-071-0/+9
|
* models: Add description and last_sync_ts columns for PackageMart Raudsepp2016-12-071-0/+4
| | | | | | | | | | | | | | | | description we will get from package.g.o per-package detailed json, last_sync_ts will be used to record when that detailed json was last pulled, so that we can rate-limit as-needed. If still using sqlite, can DROP TABLE package; and re-create with ./manage.py init or add the columns manually ALTER TABLE package ADD COLUMN description VARCHAR(500); ALTER TABLE package ADD COLUMN last_sync_ts TIMESTAMP NOT NULL; though that NOT NULL vs default on sqlalchemy's side for now might pose an issue, solving of which is an easy exercise for those that care instead of recreating.
* sync: return empty dict on projects retrieval error, so the caller won't errorMart Raudsepp2016-12-051-3/+3
|
* sync: use requests response.json() directly instead of json.loadsMart Raudsepp2016-12-041-4/+3
| | | | | | This should ensure requests will handle UTF-8 fully correctly for us Suggested-by: Doug Freed <dwfreed@mtu.edu>
* sync: Fix UTF-8 handling for projects.xml importMart Raudsepp2016-12-041-1/+1
| | | | | | | | | Need to feed response.content bytestring into ElementTree, not response.text. With the latter ET seems to figure it's already decoded and goes all latin-1 on us. From response.content bytestream it notices the UTF-8 encoding XML markup and does things right. Diagnosed-by: Doug Freed <dwfreed@mtu.edu>
* sync: Update individual maintainer names during projects sync for the time beingMart Raudsepp2016-12-041-2/+4
| | | | | ... until we don't have master data for this that we shouldn't overwrite. Also remove a now done TODO item and tweak a debug string I messed up pre-commit.
* sync: Project members and subprojects syncing to DBMart Raudsepp2016-12-041-1/+25
|
* models: Add association table and ORM relationship between projects and membersMart Raudsepp2016-12-041-0/+12
|
* sync: Initial projects syncing to DB without membersMart Raudsepp2016-12-041-7/+28
|
* models: Add Maintainer modelMart Raudsepp2016-12-041-0/+11
| | | | | As this is a new table, just re-doing "./manage.py init" should add it to db, while keeping old data.
* models: Use sqlalchemy Unicode columns instead of StringMart Raudsepp2016-12-041-4/+4
|
* Add parsed project members to the result dictMart Raudsepp2016-11-111-0/+3
|
* Normalize subproject inherit-members to True or False during parsingMart Raudsepp2016-11-101-2/+2
|
* Initial projects.xml parsing code with debug printoutMart Raudsepp2016-11-101-4/+55
|
* Add dirty sync_versions debug codeMart Raudsepp2016-11-101-1/+11
| | | | | | | This just prints the first packages versions JSON data out and exits, so just some initial debug code out of the way to sync in projects.xml first, as sync_versions will need to reference projects and maintainers, so better to finish projects.xml sync first.
* Add a full_name property to package and remove some debug spam on syncMart Raudsepp2016-09-242-1/+4
|
* Add syncing of packages in categories from packages.g.o (just name)Mart Raudsepp2016-09-071-2/+26
| | | | Also add manage.py commands to call the sync steps individually for testing
* Don't double-quote debug output for full atoms from %r usageMart Raudsepp2016-09-071-2/+2
|
* Add initial code to sync categories from packages.g.o with associated plumbingMart Raudsepp2016-09-072-0/+49
Now this should make http://localhost:5000 show the available categories: ./manage.py init ./manage.py sync_gentoo ./manage.py runserver