QED:Admin

From QED

Jump to: navigation, search

This page is intended primarily for QED system administrators.

Folder administrators may wish to review these pages:

Contents

QED Policies and Procedures

For some guidelines about changing QED access policies, see QED:Recipes.

Emergency procedures may be found in \\files\dept\ETC\Common\QED

PHP files

The PHP files for QED are in jabberwocky:~qed/mw

See also #Linux Environment below.

Blackboard

In Blackboard, the name of a course offering normally has the form COURSEID_SEMESTER (e.g. WWS572B_S2007 or ARC123-ART321_S2007). The default is to map this to Course:COURSEID (e.g. Course:WWS572B), but the mapping is governed by two arrays defined in ProjectSettings.php:

  • $wgBlackboardCoursePreserve
  • $wgBlackboardCourseToCourse

In both cases, the mapping is specified by giving the COURSEID, e.g.

  • $wgBlackboardCoursePreserve['MUS264'] = true; # use the full uppercased Blackboard id
  • $wgBlackboardCourseToCourse['WWS572B'] = 'MG'; # use MG instead of WWS572B

It would probably be better to use Course:COURSEID/SEMESTER or Course:COURSEID/YEAR rather than Course:COURSEID_SEMESTER to make navigation easier.

In any case, if the form Course:COURSEID is not used, be sure to create the appropriate #REDIRECTs.

See also: Help:Course

Implementation

See SpecialUserlogin.php and LdapAuthentication.php

Template Naming Conventions

  • Series: prefix
  • PD* for public domain works

Link Rot

Detecting dead external links

The script mw/bin/transclusionLinkrot detects linkrot resulting from the use of Template:transclude.

Managing link rot

Creating a new "folder"

Note: Access to pages with names having an embedded "/" (e.g. "PREFIX/" and "Course:XYZ/") is automatically governed by the corresponding ACL pages. That is, an ACL page becomes effective once it is "protected", whether or not the folder name has been registered.

To reduce user confusion, however, ACL pages should normally only be activated in connection with the creation of a folder as described here.

Information from the person or organization making the request

  1. The requestor should normally provide or authorize the use of an image for display as the folder's logo.
    The image that is displayed is approximately 150px x160px.
  2. Is sidebar customization required?
    Note that currently, sidebar customization implies that searching is to be restricted to pages in the folder.
    View a sidebar page such as MediaWiki:MG/sidebar for ideas.
  3. Is CSS customization required?
  4. Define access policies for the project
    • MG
    • Talk:MG
    • Project:MG
    • Project_talk:MG
  5. Can search show snippets? (ProjectSettings.php:$wgRestrictedFolders)
    • See warning about search snippets below.
  6. Is a quasi-keyboard required?
  7. Provide ACLs as required
  8. Provide information for the folder's "home page"

Warnings

  • Search Snippets
    There are currently some circumstances where users might see search snippets of pages which they cannot directly view because the search engine only checks for namespace-based and folder-based access restrictions. For example, {{Restricted:Princeton}} currently has no impact on the search engine.
  • Summaries
    The first 20 characters or so of a new article will be used as a publicly viewable "summary" if a summary has not been provided. Similarly for deleted pages. (See Article.php)

Notes

  1. If a folder in a namespace other than the main namespace is subject to folder-based read-access restrictions, then to hide search snippets, that NAMESPACE:FOLDER combination should be EXPLICITLY listed in $wgRestrictedFolders.
    This is because if the specification takes the form "FOLDER/" then the search engine simply looks at the ACLs for FOLDER in the main namespace and applies them to all other namespaces.
  2. For courses, it is usually best to use the format Course:XYZNNN where XYZNNN is as per the Princeton University Undergraduate Announcement.
    Do not use the form 'XYZ NNN'.
    For cross-listed courses, use the code of the primary listing with redirections as appropriate.
    If some other format is desired, use Course:MG as a model.

Implementation

  1. Either upload the image to be used as the logo, or install it in skins/common/images/
  2. Sidebar customization is most readily achieved by copying an existing sidebar such as MediaWiki:MG/sidebar into the similarly named sidebar page for the project, and making any modifications. See Manual:Navigation Bar for further details.
  3. Per-folder CSS customization is accomplished by adding lines of the following form to ProjectSettings.php:
    $wgFolderToStyle['FOLDER'] = 'VALUE';
    This causes the page MediaWiki:VALUE/monobook.css to be included.
    Separate lines are required for each namespace, e.g. $wgFolderToStyle['Project:FOLDER'] = 'VALUE';
  4. Update ProjectSettings.php using e.g. MG or DigitalX as a model.
    • Note especially $wgRestrictedFolders, which is used to restrict searching within folders.
  5. Create the "home page" explaining the access policies (see e.g. UC).
  6. Create the edit: and/or read: pages as required.
  7. Update QED:Projects
  8. If a quasi-keyboard is required, create MediaWiki:FOLDERNAME:edittools following the model of MediaWiki:ARA303:edittools, where FOLDERNAME is the name of the folder. Note that this currently affects all namespaces.

Administration of Folder ACLs

WikiSysop as the "superuser" can administer any folder, but QED also supports decentralized administration of each folder's read: and edit: ACL pages.

If the page FOLDER/protected: exists and is protected, then that page effectively defines the list of administrators for the folder named FOLDER. Whoever is an administrator of a folder should be told:

  1. how ACLs are constructed (see Help:ACL)
  2. that unprotecting any ACL page for a folder has the effect of removing the access restrictions which that folder defines
  3. that being able to edit a page implies being able to read it, so if someone's netid is included in the edit: page, there is no need to add it to the read: page as well.

Instructions for folder administrators are at Help:Folder administration.

.htaccess files

NOTE: The Apache .conf file is setup with "AllowOverride None" and "Allow from all" as the general rule.

This means that the default is to ignore .htaccess files.

It is therefore critical that the following directories be handled specially:

  • Globalization
  • Private
  • images
  • images/private
  • bin

Copyrighted Images

Read-access to files in the images/ directory with the copyright mark © in their name is governed by Apache.

The general principles are as follows:

  • to have read-access, one must be logged in either with an LDAP netid or a privileged id;
  • access to files with names having a prefix "FOLDER©" where "FOLDER" is the name of a folder can be further restricted.

If direct access to media files is restricted by Apache, the page QED:Restricted File is shown.

Implementation:

  • conf.d.apache2/qed.conf and/or images/.htaccess
  • <FilesMatch "©" >
  • <FilesMatch "^MG©>

To add a privileged id for FOLDER© media files, execute the following commands as "qed" from the appropriate folder:

  • cd images/Folders/FOLDER
  • htpasswd .htpasswd username

See also images/private/.htaccess

Account Management

Everyone with a Princeton netid should be able to login to QED using LDAP.

Non-LDAP accounts

If someone needs a "local" account on the QED service, it can be created using the script createUser (i.e. jabberwocky:~qed/mw/bin/createUser). To check with a user name already exists, see Special:Listusers or use the script checkUser.

To avoid collisions with LDAP, the names of such "local" user accounts must be distinguished in some manner:

  • For "read-only" accounts, using an actual email address is recommended, since currently any id with an @ in it is only granted read-access
  • Otherwise, a name of the form Prefix_userid (e.g. Uc_peak) is recommended.

Alphanumeric names may also be OK.

To reset a password, use the script in the same directory: resetPassword

These scripts only affect QED accounts. They do not affect access to media files as such access that is governed by Apache. See the section .htaccess files for details about managing access to restricted media files.

Batch Moves

moveBatch.php in the maintenance directory can be used to move several pages efficiently using MediaWiki internals. The "history" of each page will be moved as well.

The script has been modified to run in the QED environment, and to accept an option ("-s"), which will suppress the creation of redirects provided the user has the "suppressredirect" privilege.

The moveBatch.php script does not adjust internal links. In practice, therefore, moving one or more pages entails several steps, as outlined in the following subsection.

Moving a Folder

To move a folder, several steps are typically involved, as exemplified by the procedure outlined below that was used to move Course:XXXYYY to Course:XXXYYY_F2008. For convenience of exposition, the outline assumes you are logged in to jabberwocky as "qed", but the supplementary notes describe the setup that is needed.

  1. Identify the pages to be moved:
    lsQED -n Course "XXXYYY%"
  2. Check which pages have internal links that will have to be updated:
    To check all QED pages: agrepQED "Course:XXXYYY"
    To check the pages in the Course: namespace: agrep -n Course "Course:XXXYYY"
  3. Determine which links, if any, will be updated before the move, and which links will be updated after the move.
    In the present exposition, it is assumed that all such updates are done after the move.
  4. Move the pages
    php -c ~qed ~/mw/maintenance/moveBatch.php -u USERID -r F2008 -i 1 -s FILES
    where:
    • "-c ~qed" identifies the location of the php.ini file, which sets mysql.default.socket appropriately.
    • The -r option specifies a "reason" (a string) for the move.
    • USERID is a USERID (such as WikiSysop) that has sufficient privileges to read the pages to be moved.
    • FILES is the list of files to be moved, e.g. as produced by lsQED.
    • The "-s" option suppresses creating redirect pages from the old page name to the new page name.
  5. Update internal links:
    It is recommended that the script sedQED be used whereever possible to update links. Details about using sedQED are provided below. Several runs will typically be needed, e.g. to handle special cases. For example:
    echo Course:XXXYYY_F2008 | sedQED -l -e 's,Course:XXXYYY#,#,g'
    sedQED -l -e 's,Course:XXXYYY#,/#,' -e 's,Course:XXXYYY,Course:XXXYYY_F2008,g' < FILES

Other Linux Scripts

The two directories ~/bin and ~/mw/bin contain various scripts for uploading files, renaming pages, etc.

uploadToQED

The script uploadToQED can be used to upload many files at once. It is a standalone script that can be run remotely.

loginToQED

The script uploadToQED manages the login process itself, but other scripts require a cookies.txt file with suitable authentication tokens. Such a file can be easily created in the manner illustrated by loginToQED.

copyImage

The copyImage script can be used to rename one or more pages in the Image: namespace. Renaming such pages is different from renaming other types of pages because in general there will be media files that must also be renamed.

Pages in the Image: namespace can of course be renamed manually by re-uploading the media file under a new name, and then deleting the old page. This is basically the approach supported by the copyImage script. Currently, however, the copyImage script must be run on the QED server as it uses cp to copy the media file.

The main complication is that read-acess to the description page may be limited. If so, it will be necessary to provide the copyImage script with a suitable cookies.txt file, which can be done as mentioned in the section above on loginToQED.

Example

To move Image:OLDNAME to Image:NEWNAME —

  1. loginToQED
  2. copyImage --load-cookies cookies.txt OLDNAME NEWNAME
  3. Adjust any links to OLDNAME
  4. Delete Image:OLDNAME

checkURLsOnQED

This script checks that the http: and https: URLs on the rendered version of one or more QED pages actually exist.

In general, internal hyperlinks on a page can be checked using Special:Wantedpages, and external hyperlinks can be checked using maintenance/linkrot.php.

However not all URLs will appear as hyperlinks, e.g. <image src=URL>. Also, access restrictions may make the use of linkrot.php difficult.

The script checkURLsOnQED checks all the http: and https: URLs on the rendered versions of one or more QED pages. The main restriction is that only one http-user/http-passwd combination may be used at a time while checking the URLs.

sedQED

  • sedQED can be conveniently run both on jabberwocky and isaacnewton.
  • sedQED uses wikitool.pl, which expects the file wikitool.conf in the pwd. See e.g. ~qed/wikitool.conf for an example.
  • A "cookies.txt" file will also be needed. This can be created e.g. by using loginToQED or loginToQEDAsWikiSysop
  • For further details about sedQED, run: sedQED -h

Notes on wikitool:

  • wikitool requires the Perl module CMS::MediaWiki
    • In April 2009, CMS::MediaWiki was installed under jabberwocky.princeton.edu:/openpkg

watchFolder and unwatchFolder

Syntax: watchFolder NS FOLDER USERNAME

where
  • NS is the numeric number of the namespace
  • FOLDER is the page name in canonical form
  • USERNAME is the user name in canonical form.

This script will ensure that every page in the specified folder is on the watchlist of the specified user.

Example: watchFolder 0 EEB Student

unwatchFolder has the same syntax and the opposite effect.

listRecentContributions

Syntax: listRecentContributions TITLE [TIMESTAMP]

This identifies the QED pages matching TITLE% that have changed since TIMESTAMP.

Example: listRecentContributions Major_Choices 20090601

Use the --help option for details about alternative formats for TIMESTAMP.

Licenses

The menu in the Upload file form is generated using the contents of the page MediaWiki:Licenses.

Lines that begin with a * and that contain at least one vertical bar ("|") are treated specially. The basic form is:

  • templateName|Text to be displayed

If literal text is to be inserted, then use the form:

  • literal|Text to be inserted|Text to be displayed

This uses Template:literal, which accepts a single argument.

Other templates that accept a single argument may also be used, e.g. Template:self:

  • self|name of template|Text to be displayed

This results in

I, the author of this work, hereby publish it under the following license:
Template:Name of template


Linux Environment

  • Login as qed on jabberwocky.princeton.edu
  • The MySQL files for QED are in ~qed/mysql5/ (e.g. the database is in TigerWeb/ and the socket is var/mysql.sock).
  • The standard reboot process will restart Apache, but, as of this writing, mysql5 for the QED service must be started separately.
    • This is handled by a cron job called nanny
      • ~/bin/nanny checks that mysql5 is running every three minutes.
      • output goes to /u/qed/www/logs/nanny_log
  • batch jobs (atq)
    • ~/bin/my.atq will list the currently scheduled batch jobs
      Ccurrently there are 8, two for each task in the table below.
      The log file is /u/qed/www/logs/batch.log
Task Command Comment
hotcopy database (evening) /u/qed/bin/batch.hotcopyTigerWeb (*)
hotcopy database (morning) /u/qed/bin/batch.am.hotcopyTigerWeb (*)
archive logs /u/qed/bin/batch.stash-qed-logs See ~/www/logs/arc
capture ClustrMap /u/qed/mw/bin/batch.getClustrMaps Updates ~/mw/public/maps-clusters (see below)

(*) Check ~/TigerWeb/dump periodically. Normally, the backups are copied to isaacnewton.princeton.edu:~peak/qed/TigerWeb and then erased from jabberwocky. The backups on isaacnewton.princeton.edu should be periodically erased.

ClustrMap

See ~/mw/public/maps-clusters/README for details about updating Image:Mapping Globalization.

File Uploads

Suffixes

To change the list of permitted file extensions, change $wgFileExtensions in LocalSettings.php and change the list on the Help:Upload page. It may also be necessary to add MIME-type information as described in the subsection below.

The list of allowed file extensions presented on the "Upload file" form and elsewhere in the online help is dynamically generated, exceept on the Help:Upload page as noted above.

MIME Types

  • MIME types for file uploads:
    • See ~qed/mw/includes/mime.types
  • MIME types for file downloads:
    • If a new file suffix is not defined in the Apache mime.types file (~qed/www/conf/mime.types), add an "AddType" specification to ~qed/www/conf.d.apache2/qed.conf.

Maximum Size

There are several variables governing how large a file can be for it to be uploaded successfully. The main ones are the PHP configuration variables post_max_size and upload_max_filesize (~/www/conf/php.ini). Their current values can be see at QED:Variables#Maximum_Size_of_an_Uploaded_File.

Other variables that may be relevant include:

  • The PHP configuration variables memory_limit, max_execution_time, and max_input_time
    • memory_limit should not be too large, otherwise concurrent uploads of large files will exhaust available memory.
  • Apache's "LimitRequestBody" (see ~/www/conf.d.apache2/php.conf)

See also What affects the maximum file size that can be uploaded?.

robots.txt

As of 2007-11-09, jabberwocky:/u/qed/mw/robots.txt contains these lines to prevent Google from indexing the wrong pages: 

User-agent: *
Disallow: /*&printable=yes
Disallow: /*&redirect=no


Provisional Features

Template:Folder:FOLDER

Selected pages in the Image: namespace can be made to appear as though they are in a defined folder.

Specifically, if a folder named FOLDER has been defined, then a page in the Image: namespace will appear as though it were in the folder if the template Folder:FOLDER is included on the page.

For example, for the MG folder, the following text would be added: {{Folder:MG}}

At most one such template should be specified on any given page in the Image: namespace.

This feature is used for the Mapping Globalization project.

Implementation: index.php uses pageUsesTemplate defined in setup.php

Restricting Access to Media Files based on Course Enrollment

Access to media description pages with names of the form Image:FOLDER©STRING (where STRING is an arbitrary string) can be restricted using ACL pages in the Course: namespace. That is, access to Image:FOLDER© pages depends on the protected status and contents of the pages: Course:FOLDER/read:, Course:FOLDER/edit: and Course:FOLDER/protect:.

In a nutshell, whoever can read Course:FOLDER/read: can also read Image:FOLDER© pages, and similarly for editing.

Access to media files with a copyright symbol © in their names is restricted by the web-server to users with Princeton netids.

Implementation:

  • web server:
    • qed.conf
    • /u/qed/mw/images/Folders/MG/.htpasswd
  • MediaWiki:
    • FolderBasedAccess::hasCourseCopyrightPermission

For Testing

QED:PT

Experiments

 QED:experiment
 Wiki Farm

See also


To email another QED user assuming your own email address is valid, use the form Special:Emailuser/SENDEE

Additional Topics

This page has 5 sub-pages:

Retrieved from "http://qed.princeton.edu/main/QED:Admin"
Personal tools