Page MenuHomeNicheWork Phabricator

MABS: Sync ONE article between asywiki demo wikis
Open, HighPublic

Description

Tomorrow I should start syncing ONE article between the wikis and, once that works, expand to a group of articles.

Details

Deadline
Sep 4 2019, 12:00 AM

Event Timeline

hexmode created this task.Sep 3 2019, 10:35 AM
hexmode triaged this task as Normal priority.
hexmode added a comment.EditedSep 12 2019, 11:43 AM

Command to use, from instructions:

$ git clone -c remote.origin.pages='Atlanta' mediawiki::http://asyncwiki-moon.wmflabs.org/demo
hexmode added a comment.EditedSep 12 2019, 1:00 PM

After fixing some bugs related to fetching single pages it looks like I need to run rebuild recent changes. since the api call is returning empty.

feh:

$ sudo -u www-data WIKI=demo php maintenance/rebuildrecentchanges.php --from 20190101000000 --to 20191201000000
Rebuilding range of 28857600 seconds (334 days)
Clearing recentchanges table for time range...
Loading from page and revision tables...
Inserting from page and revision tables...
[cefdc151b8d10c6a08a37a8e] [no req]   InvalidArgumentException from line 713 of /opt/htdocs/mediawiki/includes/user/User.php: Cannot create a user with no name, no ID, and no actor ID
Backtrace:
#0 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(151): User::newFromAnyId(string, string, NULL)
#1 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(70): RebuildRecentchanges->rebuildRecentChangesTablePass1(Wikimedia\Rdbms\LBFactorySimple)
#2 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(94): RebuildRecentchanges->execute()
#3 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(519): require_once(string)
#4 {main}

Switched to REL1_33 and tried:

# sudo -u www-data WIKI=demo php maintenance/rebuildrecentchanges.php --from 20190101000000 --to 20191201000000
Rebuilding range of 28857600 seconds (334 days)
Clearing recentchanges table for time range...
Loading from page and revision tables...
Inserting from page and revision tables...
Updating links and size differences...
Loading from user and logging tables...
Flagging bot account edits...
Removing duplicate revision and logging entries...
Done.
[82ee5cb68df777ec8729a708] [no req]   Error from line 264 of /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort()
Backtrace:
#0 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(104): Wikimedia\Rdbms\LBFactory->commitMasterChanges(string)
#1 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(519): require_once(string)
#2 {main}
PHP Fatal error:  Uncaught Error: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort() in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php:264
Stack trace:
#0 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(183): Wikimedia\Rdbms\LBFactory->commitMasterChanges('Wikimedia\\Rdbms...')
#1 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(156): Wikimedia\Rdbms\LBFactory->shutdown(0)
#2 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(734): Wikimedia\Rdbms\LBFactory->destroy()
#3 [internal function]: Wikimedia\Rdbms\LBFactory->__destruct()
#4 {main}
  thrown in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php on line 264

Fatal error: Uncaught Error: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort() in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php:264
Stack trace:
#0 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(183): Wikimedia\Rdbms\LBFactory->commitMasterChanges('Wikimedia\\Rdbms...')
#1 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(156): Wikimedia\Rdbms\LBFactory->shutdown(0)
#2 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(734): Wikimedia\Rdbms\LBFactory->destroy()
#3 [internal function]: Wikimedia\Rdbms\LBFactory->__destruct()
#4 {main}
  thrown in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php on line 264

Trying update.php (currently running)

hexmode added a comment.EditedSep 12 2019, 1:59 PM

got a few errors like this:

User name "Ensmelle" is usable, cannot create an anonymous actor for it. Run maintenance/cleanupUsersWithNoId.php to fix this situation.

Did that, and it modified no rows. Same error when running update.php. googling.

Switched back to REL1_32 and tried

# sudo -u www-data WIKI=demo php maintenance/migrateActors.php --force
...cannot update while $wgActorTableSchemaMigrationStage lacks SCHEMA_COMPAT_WRITE_NEW

Did that now we're seeing this: in a bad state due to production tables being imported where Reedy suggests insert ignore brefore abandoning because it was somehow "fixed".

Apparently it fixed itself without my knowledge. Now getting what looks like this bug.

fixed by adding a row with role_id=2 and role_name='aux'

php maintenance/populateContentModel.php --ns all --table revision

keep ending up here:

Refreshing links tables.
Starting from page_id 1 of 173884.
[b0f10473827284f0a2844f14] [no req]   MediaWiki\Revision\RevisionAccessException from line 1635 of /opt/htdocs/mediawiki/includes/Revision/RevisionStore.php: Main slot of revision 934 not found in database!
Backtrace:
#0 /opt/htdocs/mediawiki/includes/Revision/RevisionStore.php(1671): MediaWiki\Revision\RevisionStore->loadSlotRecords(string, integer)
#1 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}()
#2 /opt/htdocs/mediawiki/includes/Revision/RevisionSlots.php(165): call_user_func(Closure)
#3 /opt/htdocs/mediawiki/includes/Revision/RevisionRenderer.php(169): MediaWiki\Revision\RevisionSlots->getSlots()
#4 /opt/htdocs/mediawiki/includes/Revision/RevisionRenderer.php(128): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#5 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#6 /opt/htdocs/mediawiki/includes/Revision/RenderedRevision.php(175): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#7 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1266): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#8 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1288): MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
#9 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1560): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
#10 /opt/htdocs/mediawiki/includes/page/WikiPage.php(2106): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
#11 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(275): WikiPage->doSecondaryDataUpdates(array)
#12 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(198): RefreshLinks::fixLinksFromArticle(integer, boolean)
#13 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(83): RefreshLinks->doRefreshLinks(integer, boolean, string, boolean, boolean)
#14 /opt/htdocs/mediawiki/maintenance/rebuildall.php(60): RefreshLinks->execute()
#15 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(94): RebuildAll->execute()
#16 /opt/htdocs/mediawiki/maintenance/rebuildall.php(67): require_once(string)
#17 {main}

Note that the above was resolved by re-importing the SQL and then running rebuildrecentchanges.php

hexmode shifted this object from the Restricted Space space to the S3 Public NicheWork space.Sep 16 2019, 9:45 AM
hexmode changed the visibility from "All Users" to "Public (No Login Required)".
hexmode raised the priority of this task from Normal to High.Sep 16 2019, 9:55 AM

Wiki is kind of working, so I'm gonna stop fiddling with it. Trying the original git clone:

$ . ~/work/code/mediawiki-git-remote/setup.sh
$ git clone -c remote.origin.pages='Atlanta' mediawiki::https://asyncwiki-moon.wmflabs.org/demo .
Cloning into '.'...
1 pages found.
1/1: Revision #3818512 of Atlanta
Wide character in print at /usr/lib/x86_64-linux-gnu/perl/5.28/IO/Handle.pm line 157.
1/1: Revision #3818512 of Atlanta
Wide character in print at /usr/lib/x86_64-linux-gnu/perl/5.28/IO/Handle.pm line 157.
$ ls
Atlanta.mw

Leaving those warnings for now. Now to edit and push the edit.

Created a bot user user Special:BotPasswords and pushed an edit back to the moon. Voila!

$ git remote rename origin moon
$ git remote add earth mediawiki::https://asyncwiki-earth.wmflabs.org/demo

Rebuilding on earth:

$ sed -i 's#/..wgDBTableOptions./#ENGINE=InnoDB, DEFAULT CHARSET=binary#' init_1.27.sql 
$ mysqladmin drop -f wiki_demo; mysqladmin create wiki_demo; (echo "set names 'utf8';"; echo "set character set utf8;";echo 'set unique_checks = 0;'; echo 'set autocommit = 0;'; cat init_1.27.sql ;echo "alter table page drop index name_title;"; pv 01-enwikivoyage.sql.bz2 | bzcat ; echo 'set unique_checks = 1;' ) | mysql wiki_demo