Page MenuHomeNicheWork Phabricator
Create Task
Maniphest T326

MABS: Sync ONE article between asywiki demo wikis
Open, HighPublic
Actions

Description

Tomorrow I should start syncing ONE article between the wikis and, once that works, expand to a group of articles.

Details

Deadline
Sep 4 2019, 12:00 AM

Related Objects
Search...

Event Timeline

hexmode created this task.Sep 3 2019, 10:35 AM
hexmode triaged this task as Normal priority.
hexmode added a comment.EditedSep 12 2019, 11:43 AM

Command to use, from instructions:

$ git clone -c remote.origin.pages='Atlanta' mediawiki::http://asyncwiki-moon.wmflabs.org/demo
hexmode added a comment.EditedSep 12 2019, 1:00 PM

After fixing some bugs related to fetching single pages it looks like I need to run rebuild recent changes. since the api call is returning empty.

hexmode added a comment.Sep 12 2019, 1:04 PM

feh:

$ sudo -u www-data WIKI=demo php maintenance/rebuildrecentchanges.php --from 20190101000000 --to 20191201000000
Rebuilding range of 28857600 seconds (334 days)
Clearing recentchanges table for time range...
Loading from page and revision tables...
Inserting from page and revision tables...
[cefdc151b8d10c6a08a37a8e] [no req]   InvalidArgumentException from line 713 of /opt/htdocs/mediawiki/includes/user/User.php: Cannot create a user with no name, no ID, and no actor ID
Backtrace:
#0 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(151): User::newFromAnyId(string, string, NULL)
#1 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(70): RebuildRecentchanges->rebuildRecentChangesTablePass1(Wikimedia\Rdbms\LBFactorySimple)
#2 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(94): RebuildRecentchanges->execute()
#3 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(519): require_once(string)
#4 {main}
hexmode added a comment.Sep 12 2019, 1:54 PM

Switched to REL1_33 and tried:

# sudo -u www-data WIKI=demo php maintenance/rebuildrecentchanges.php --from 20190101000000 --to 20191201000000
Rebuilding range of 28857600 seconds (334 days)
Clearing recentchanges table for time range...
Loading from page and revision tables...
Inserting from page and revision tables...
Updating links and size differences...
Loading from user and logging tables...
Flagging bot account edits...
Removing duplicate revision and logging entries...
Done.
[82ee5cb68df777ec8729a708] [no req]   Error from line 264 of /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort()
Backtrace:
#0 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(104): Wikimedia\Rdbms\LBFactory->commitMasterChanges(string)
#1 /opt/htdocs/mediawiki/maintenance/rebuildrecentchanges.php(519): require_once(string)
#2 {main}
PHP Fatal error:  Uncaught Error: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort() in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php:264
Stack trace:
#0 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(183): Wikimedia\Rdbms\LBFactory->commitMasterChanges('Wikimedia\\Rdbms...')
#1 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(156): Wikimedia\Rdbms\LBFactory->shutdown(0)
#2 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(734): Wikimedia\Rdbms\LBFactory->destroy()
#3 [internal function]: Wikimedia\Rdbms\LBFactory->__destruct()
#4 {main}
  thrown in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php on line 264

Fatal error: Uncaught Error: Call to undefined method Wikimedia\ScopedCallback::newScopedIgnoreUserAbort() in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php:264
Stack trace:
#0 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(183): Wikimedia\Rdbms\LBFactory->commitMasterChanges('Wikimedia\\Rdbms...')
#1 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(156): Wikimedia\Rdbms\LBFactory->shutdown(0)
#2 /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php(734): Wikimedia\Rdbms\LBFactory->destroy()
#3 [internal function]: Wikimedia\Rdbms\LBFactory->__destruct()
#4 {main}
  thrown in /opt/htdocs/mediawiki/includes/libs/rdbms/lbfactory/LBFactory.php on line 264
hexmode added a comment.Sep 12 2019, 1:58 PM

Trying update.php (currently running)

hexmode added a comment.EditedSep 12 2019, 1:59 PM

got a few errors like this:

User name "Ensmelle" is usable, cannot create an anonymous actor for it. Run maintenance/cleanupUsersWithNoId.php to fix this situation.

Did that, and it modified no rows. Same error when running update.php. googling.

hexmode added a comment.Sep 12 2019, 3:09 PM

Switched back to REL1_32 and tried

# sudo -u www-data WIKI=demo php maintenance/migrateActors.php --force
...cannot update while $wgActorTableSchemaMigrationStage lacks SCHEMA_COMPAT_WRITE_NEW
hexmode added a comment.Sep 12 2019, 3:27 PM

Did that now we're seeing this: in a bad state due to production tables being imported where Reedy suggests insert ignore brefore abandoning because it was somehow "fixed".

hexmode added a comment.Sep 12 2019, 6:09 PM

Apparently it fixed itself without my knowledge. Now getting what looks like this bug.

hexmode added a comment.Sep 12 2019, 6:39 PM

fixed by adding a row with role_id=2 and role_name='aux'

hexmode added a comment.Sep 12 2019, 7:23 PM

php maintenance/populateContentModel.php --ns all --table revision

hexmode added a comment.Sep 12 2019, 7:57 PM

keep ending up here:

Refreshing links tables.
Starting from page_id 1 of 173884.
[b0f10473827284f0a2844f14] [no req]   MediaWiki\Revision\RevisionAccessException from line 1635 of /opt/htdocs/mediawiki/includes/Revision/RevisionStore.php: Main slot of revision 934 not found in database!
Backtrace:
#0 /opt/htdocs/mediawiki/includes/Revision/RevisionStore.php(1671): MediaWiki\Revision\RevisionStore->loadSlotRecords(string, integer)
#1 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}()
#2 /opt/htdocs/mediawiki/includes/Revision/RevisionSlots.php(165): call_user_func(Closure)
#3 /opt/htdocs/mediawiki/includes/Revision/RevisionRenderer.php(169): MediaWiki\Revision\RevisionSlots->getSlots()
#4 /opt/htdocs/mediawiki/includes/Revision/RevisionRenderer.php(128): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#5 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#6 /opt/htdocs/mediawiki/includes/Revision/RenderedRevision.php(175): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#7 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1266): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#8 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1288): MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
#9 /opt/htdocs/mediawiki/includes/Storage/DerivedPageDataUpdater.php(1560): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
#10 /opt/htdocs/mediawiki/includes/page/WikiPage.php(2106): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
#11 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(275): WikiPage->doSecondaryDataUpdates(array)
#12 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(198): RefreshLinks::fixLinksFromArticle(integer, boolean)
#13 /opt/htdocs/mediawiki/maintenance/refreshLinks.php(83): RefreshLinks->doRefreshLinks(integer, boolean, string, boolean, boolean)
#14 /opt/htdocs/mediawiki/maintenance/rebuildall.php(60): RefreshLinks->execute()
#15 /opt/htdocs/mediawiki/maintenance/doMaintenance.php(94): RebuildAll->execute()
#16 /opt/htdocs/mediawiki/maintenance/rebuildall.php(67): require_once(string)
#17 {main}
hexmode added a comment.Sep 15 2019, 10:57 PM

Note that the above was resolved by re-importing the SQL and then running rebuildrecentchanges.php

hexmode shifted this object from the Restricted Space space to the S3 Public NicheWork space.Sep 16 2019, 9:45 AM
hexmode changed the visibility from "All Users" to "Public (No Login Required)".
hexmode raised the priority of this task from Normal to High.Sep 16 2019, 9:55 AM
hexmode added a comment.Nov 5 2019, 2:42 PM

Wiki is kind of working, so I'm gonna stop fiddling with it. Trying the original git clone:

$ . ~/work/code/mediawiki-git-remote/setup.sh
$ git clone -c remote.origin.pages='Atlanta' mediawiki::https://asyncwiki-moon.wmflabs.org/demo .
Cloning into '.'...
1 pages found.
1/1: Revision #3818512 of Atlanta
Wide character in print at /usr/lib/x86_64-linux-gnu/perl/5.28/IO/Handle.pm line 157.
1/1: Revision #3818512 of Atlanta
Wide character in print at /usr/lib/x86_64-linux-gnu/perl/5.28/IO/Handle.pm line 157.
$ ls
Atlanta.mw

Leaving those warnings for now. Now to edit and push the edit.

hexmode added a comment.Nov 5 2019, 3:01 PM

Created a bot user user Special:BotPasswords and pushed an edit back to the moon. Voila!

hexmode added a comment.Nov 9 2019, 3:25 PM
$ git remote rename origin moon
$ git remote add earth mediawiki::https://asyncwiki-earth.wmflabs.org/demo
hexmode added a comment.Nov 11 2019, 2:18 PM

Rebuilding on earth:

$ sed -i 's#/..wgDBTableOptions./#ENGINE=InnoDB, DEFAULT CHARSET=binary#' init_1.27.sql 
$ mysqladmin drop -f wiki_demo; mysqladmin create wiki_demo; (echo "set names 'utf8';"; echo "set character set utf8;";echo 'set unique_checks = 0;'; echo 'set autocommit = 0;'; cat init_1.27.sql ;echo "alter table page drop index name_title;"; pv 01-enwikivoyage.sql.bz2 | bzcat ; echo 'set unique_checks = 1;' ) | mysql wiki_demo