Aleph v18 Cataloging

Indexing Queue

We get questions every once in a while along the lines of "Where's my record?!" that turns out to be a delay indexing. When we get reports/questions like this:

The first thing we do at FCLA is check the indexing queue. If a new record is loaded but not yet indexed it looks like it wasn't loaded at all. (New records can be retrieved by OCLC number before they get completely indexed).

What probably is going on is that your record is sitting in the queue of records waiting to index. Creating or updating a bib, holding, item, or authority record can send a bib to indexing. The indexing queue can get pretty large because of activities such as a large batch load of MARC records, a large GenLoad, a large global change, or an authority load that sent a lot of bib records to indexing.

Aleph assigns priorities to different types of record updates and batch activities, then processes them in priority order. Here's a very general picture of the order of indexing:
1. new records
2. records updated online or through the OCLC server
3. records updated through batch services such as global change or
through LCA10 loads
The priorities can be tweaked. For example, by default new records loaded through GenLoad fall into group #1 (highest priority). If you check "lower indexing priority" in the load profile, the records are assigned a priority between #2 and #3.

The mechanism for assigning priority is adjustment of the date/time stamp for a record in the indexing queue. Records with priority 1 are assigned the year 1990. Records with priority 2 are assigned the year 1998. Records with priority 3 are assigned the actual year (currently 2008). The record with the earliest date and time indexes first. A new record created right now jumps to the head of the indexing queue with the other new records and ahead of a record changed online five minutes ago.

If you can't retrieve your record with a search, the bib is probably sitting in the indexing queue.

ARROW INDEXING QUEUE REPORT
There's an ARROW report that shows the status of the indexing queue called Check Indexing Queue. You can run this yourself to see why your record may be delayed.

You can check:
a) Total number of records in an institution's queue. This doesn't tell you anything about priority. For example, while this week's LCA10 load sent lots of bibs to indexing, these bibs have the lowest indexing priority; your new records and changes made in the GUI client should index first.
b) First 20 records in the queue, which can tell you what's about to index.
c) First 1000 records in the queue.
d) Whether a specific record is in the queue waiting to index.

The information reported is limited: the bib number, the date/time stamp which determines indexing priority, and what bib, holding, item, or authority record create/update sent the bib to indexing. Here are sample entries from "first 20" reports:

Sample from a USF report:
DATE/TIMESTAMP
BIB NUMBER FOR PRIORITY SENT TO INDEXING FROM
001275889 200809040921359 SFU01001275889
001276509 200809040921359 SFU01001276509

Sample from a UF report showing one record assigned priority 2, and 2 records sent to indexing by a holdings create/update:
DATE/TIMESTAMP
BIB NUMBER FOR PRIORITY SENT TO INDEXING FROM
004183163 199809041457563 UFU01004183163
004146910 200809040926018 UFU60004970373UFU01004146910
004146933 200809040928323 UFU60004970374UFU01004146933


INDEXING PROCESS IS HUNG
Occasionally the indexing process (ue_01) can hang and nothing indexes at all. Signs of this:
ARROW report of total number of records in the queue doesn't change or goes up when run several times over several minutes; and: 
ARROW report of first 20 records doesn't change when run several times over several minutes.

If you're familiar with the staff web page, you can see a better indicator of whether ue_01 is hung. Go into view/download files and check the bib library's scratch directory (e.g., sfu01/scratch, nfu01/scratch). Look for the most recent run_e_01... file. Refresh the display several times over several minutes. If there are records in your indexing queue and the run_e_01... file size and date/time stamp don't change, your ue_01 is probably hung. Contact FCLA and we'll restart your process.


New in Version 18 Cataloging

New Client for Cataloging – separate Items, Search and Task Manager modules no longer exist.


Online Help – Reorganization

Default Client Settings – GuiSys.ini file - Removed from ALEPHCOM.INI so that certain defaults can be maintained if ALEPHCOM.INI is updated with newer version.

Cataloging Record – The cataloging editor can now handle documents with up to 500 lines when each subfield is held on a different line (old limit, 2000). Each line is limited to 2000 characters. The limit of 45,000 characters per record remains. If the record exceeds 45,00 characters an error message will display and the operation is halted.

Record Manager – Displays information about the record in the editor view. Related records can be seen in an expandable tree view.


URL Checker – Services Menu Option – Batch service for checking the validity of URLs.

Locate – You can now select multiple records in the results list of the "Locate Similar" record feature. Each selected record will load into a separate record editor window.

ISSN – ISSNs are now validated when a record is saved to the server just as ISBNs.

UNICODE Storage – Unicode is now stored in UTF-, formerly stored in US7ASCII.

UNICODE Tool Tip – It is now possible to view the Unicode value of a character in the Catalog Editor window by placing the mouse pointer over the character in the record: after about two seconds a tool tip appears above the character, containing a string composed of "U+" followed by a four digit hexadecimal value, e.g., "U+0041" = A.

View in OPAC – While record is displayed in the editor view, staff can now view the Web OPAC version of the record being displayed in the Cataloging module – Lower right pane below record editor, third tab (Browser).

New Fix Routines

ITEMS

Smart Barcode Export Routine Service

INDEX BUILDING

Optimization and structural changes in index building – for logical bases, words and headings

Building indexes in parallel to a running system (PID)

AUTHORITIES

Authority Recheck for selected headings – Browse List in Cataloging

Conflicting Headings Improvement – The "Categories" Mechanism

Correction of Tags and Indicators (for the MARC21 format)

LOADERS

OCLC Batch Loader –It is now possible to load OCLC records using an Aleph batch process.

Loader logger – The system logs all loads from OCLC and MARCIVE – A log report can be produced by using a service in the client (manage-94).