Batch Processes introduction & links

_{First very rough draft of general links and tasks useful to Batch & Sierra regularly occurring duties}

• ^{started 9/17/2023; updating continually}

General Principles

Most batch files are for various e-resources, but some tangible batch record collection are also loaded via batch files ((print DDA files), vendor ready files, government document and map files, (loaded by Government Information staff), etc.)

1. Copy/Paste the batch file name(s) into each Collection Tracker in Box as soon as they are downloaded/received

2. Take note of how many records there are in a batch file

   • if that number changes after a batch processes, then something went wrong, and the file or the process needs to be checked for problems: some of which can be as simple as duplicate bibs in the file, but at other times, may be indicating one or more bibs with errors from a typo or a missing field)

3. When loading into Sierra, always click Use Review Files; then Test the batch, and if it looks good, Load.

4. Avoid at all costs, loading files out of order (generally, this is by the date in the file name but some dates will need to be added as you are downloading them)

   • If possible, keep up with weekly loads, so loading dates out of order does not become a possibility

   • For most collection batch files, when the files have the same date; first load new/additions files, then delete files (there are exceptions noted in the these workflows); but, always check the date, and load files with earliest date before the latest.

5. 001 fields: each MARC record in a batch file needs a unique control number that is not duplicated anywhere in the catalog (we may need to overlay them later) - some collections already have good 001 as as supplied by the vendor; and all current batch collections have documented methods to apply prefixes or create completely unique numbers.

   • For any completely new batch collection, where the 001 fields are missing or contain simple numbers, there will need to be a method worked out to apply a new, non-duplicating prefix or a creation method for a 001 (and add it to your new workflow)

6. Hooks: a series of fields and codes are applied to each batch collection, making them easier to batch edit or remove. The current master document of hooks is kept in USU’s Box; updates will need to be added and updates distributed to various areas of the library (such cataloging, LIT, etc.), as old batch collections are changed, deaccessioned, or new ones are added. The most current Master List E-Resource Hooks is kept here: https://usu.app.box.com/file/1326778691763.

   • It will need regular updating, with new batch purchases, and regular batch perpetual purchases, as seen in the screen shot below:

Batch/Sierra list of tasks:

• Original Systems Librarian’s list of tasks (Discovery Services Librarian list will be much more focused on records large-bulk purchases or Sierra weeding/moving/batch cataloging projects):

• Working Schedule of Tasks for Discovery Services Librarian (under revision) in Box: https://usu.app.box.com/file/1321320586852

Cheryl’s plan of training:

Start with single record loads for Wageningen, then weekly record loads of Cambridge EBA, Taylor & Francis EBA, Safari/O'Reilly subscription, and the YBP discovery records, and then the more complex Academic Complete update records.  (One thing to remember is to watch the dates on the ebook discovery and deletes and make sure you load them in order; also, with 1 exception, don't load deletes until you have loaded the discovery records with earlier dates.) Make sure you use the tracking sheet versions with “current” at the end.

Preliminary list to start with:

• Wageningen: cataloged individually & tracked against the purchase invoice & Wageningen site itself.

  • workflow:

• Cambridge EBA: https://usu.app.box.com/file/1318202855178

  • workflow:

• Taylor & Francis EBA: https://usu.app.box.com/file/1318195064664

  • workflow:

• Safari/O’Reilly subscription: https://usu.app.box.com/file/1318206846659

  • workflow

• YBP discovery records(and deletes): https://usu.app.box.com/file/1321317102096

  • workflow

• pDDA deletes via email from Kevin and Tyler (Cheryl hasn’t tracked; I’ve been just keeping stats)

  • workflow

• Streaming videos from Gaby: https://usu.app.box.com/file/1323615493212 (keep up with regular emails of adds/deletes)

  • workflow started here:

• Academic Complete update records--part of https://usu.app.box.com/file/1321317102096

  • workflow

• University Press of Colorado ebooks: https://usu.app.box.com/file/1321432292725

  • workflow

• Follett (Moore books; create lists used to create order recs by Acq staff)

  • workflow:

• other more difficult batch processing

• Further note from Cheryl: just fyi, I'm trying to include any tracking files you might need eventually.  A lot of what I'm adding currently are NOT pressing and could be ignored until you have a day where you want a challenge. 

• WEST archived and non-archived yearly uploads,

  • annual regular workflow

  • archived workflow

• Sierra weeding and batch cataloging projects

  • sample workflow uploaded:

  • see also batch projects

The former Batch systems librarian’s (Cheryl)( documentation and Tracking of files will remain in Box

• here: https://usu.app.box.com/folder/227209648067?s=q531024vtone9ogkwa6cv2uh41sq56qx

Statistics for new Discovery Services Librarian (temp. and new position) going forward:

From: Liz Woolcott

To: Melanie Shaw; Becky Skeen

>>>>>>>

Yes, Cheryl was reluctant to put stats in our joint base – mostly because she kept them differently.  I am very appreciative that you are thinking about how to build them in.  It has been the missing piece we have needed for a long while to demonstrate the cataloging work done in the library.

I’ve included my suggested answers to your questions below in red. Becky, what do you think?

Thanks again for all your deep thinking on this Melanie!

Liz

From: Melanie Shaw <melanie.shaw@usu.edu>
Sent: Thursday, November 2, 2023 2:50 PM
To: Liz Woolcott <liz.woolcott@usu.edu>; Becky Skeen <becky.skeen@usu.edu>
Subject: Discovery Services batch statistics

I asked Cheryl about this, but it turns out she did not keep these kind of stats (might have been better if she had).

I have been keeping my own statistics of all the batch files, which come in New, Overlay (changes), and Deletes, usually (with the Deletes being just a variation of the Overlay load, that I then use to Batch Delete the records – usually “discovery” eBooks that we never owned, from the patron driven delivery loads.

 So, there’s several questions I have about what statistics the Discovery Services Librarian (and me, in the meantime) should keep.

First, I have been counting the Delete loads as records loaded, but I then remove them.  So, maybe a new category? Or possibly not, since we no longer base our numbers of things in the catalog from our cataloging stats. Yes!  These definitely need to be a different category.  You can use the “MARC Batch-Deleted Records” in the stats.  That is what Carol has been using for her batch removed records.

I also have gotten a steady stream of emails with single deletions to be done manually, where I simply erase a PDDA record that we have gotten in another format. I’ve counted those as part of the batch stuff, since again, I haven’t cataloged them individually (just deleted them).  This one is a hard one.  The closest thing we have is the “Sierra-Items Deleted” column.  A while back, Barb made the call to collapse the “titles deleted” and the “items deleted” into one category to make it easier to record weeding projects.  We decided the larger number was probably more useful.  The PDDA records are odd because we never owned the item, it was always just a discovery record.  So, you have a few options to consider: 1) Could make a new category for “records deleted” so that the PDDA number could be reported more accurately (but it might be complicated by the fact that Barb doesn’t report records deleted for her work – and assumptions might be made that this field includes all records deleted) or 2) you could use the “Sierra-Items Deleted” for these few one offs (complication is that this could be interpreted as materials deaccessioned)  or 3) could create its own category for “PDDA records deleted” or 4) Option I can’t think of at the moment… 😊

 I also download individual records from the streaming video sites for Gaby, again these are vendor records, and I edited them the same as the batch records. So, I could count them as individually cataloged titles, but I don’t do much check (except for the link) and I add in the “hooks”, so I’ve been uncertain how to count them. Right now they are in my Batch stats, because, vendor records and process is the same. I would recommend “Genstacks-CopyCat-Electronic” for this.  As long as you are doing them one-by-one that is.  If you are pulling them individually but compiling them into groups to manipulate and load records, then “GenStacks-Batch-Modified”.

Melanie

I will add Statistics to each workflow as soon as possible (while still keeping weekly duties on track…)

FTP downloads software/app:

Since all browsers have stopped supporting FTP protocols, we will use LIT’s suggestion of the WinSCP software:

• Download for Windows: https://winscp.net/eng/download.php

  • All users

  • Accept all the defaults and lawyer repellant

  • and Open

• type in the vendor’s ftp site address and log in with user name & password:

• 

  A split window will open with your computer on the left; the vendors files on the right:

For historical use & older tracking:

Trello website: Historical procedures & notes, but contains earliest logs of Batch record loads

https://trello.com/b/CWYisa6s/cheryl-and-melanie-batch-record-loading