Skip to content

feat(importer): added new gobi importer - #1116

Merged
kpsherva merged 1 commit into
CERNDocumentServer:masterfrom
callumlewis05:new-importer-provider
Aug 31, 2026
Merged

feat(importer): added new gobi importer#1116
kpsherva merged 1 commit into
CERNDocumentServer:masterfrom
callumlewis05:new-importer-provider

Conversation

@callumlewis05

Copy link
Copy Markdown
Contributor

Added new importer provider for GOBI marcxml.

image

Works with given test file.

Written to this specification to make it easier to review

PROCESSED FIELDS:

Required:
001	Control Number -> provider_recid and alternative_identifiers (scheme "GOBI")
003	Control Number Identifier -> agency_code
100 $a  Main Entry-Personal Name - Personal Name -> author_name
245 $a	Title Statement	- Title -> title
245 $b	Title Statement	- Remainder of Title -> title
264 $a  (ind2=1)  Production, Publication, Distribution, Manufacture, and Copyright Notice - Place of production, publication, distribution, manufacture -> 
264 $b  (ind2=1)  Production, Publication, Distribution, Manufacture, and Copyright Notice - Name of production, publisher, distributer, manufacturer

EItem:
856 $u  (ind1=4, ind2=0) Uniform Resource Identifier -> eitem.urls

Optional:
020 $a	International Standard Book Number - International Standard Book Number -> identifiers
050 $a	(ind2=4) Library of Congress Call Number - Classification Number -> subjects (scheme LOC)
082 $a  (ind2=4) Decimal Classification Number - Classification number -> subjects (scheme DEWEY)
082 $a  (ind1=0, ind2=4) Decimal Classification Number - Classification number -> subjects (scheme DEWEY)
264 $c  (ind2=1)  Production, Publication, Distribution, Manufacture, and Copyright Notice - Date of production, publication, distribution, manufacture, or copyright notice -> publication_year
300 $a  Physical Description - Extent -> number_of_pages
490 $a  Series Statement - Series Statement -> _serial.title
500 $a	General Note - General Note -> note
505 $a  Formatted Contents Note - Formatted Contents Note -> table_of_content
505 $t  Formatted Contents Note - Title -> table_of_contents
650 $a  (ind2=0) Subject Added Entry-Topical Term - Topical term or geographic name entry element -> keywords
700 $e  Added Entry-Personal Name - Personal name -> author_name
700 $e  Added Entry-Personal Name - Relator term -> roles
776 $w  (ind1=0, ind2=8) Additional Physical Form Entry - International Standard Book Number -> identifiers (material PRINT_VERSION)
776 $z  Additional Physical Form Entry - International Standard Book Number -> identifiers (material PRINT_VERSION)
830 $v  Series Added Entry-Uniform Title - Volume/sequential designation -> _serial.volume



IGNORED FIELDS:

005	Date and Time of Latest Transaction
006	Fixed-Length Data Elements - Additional Material Characteristics
007	Physical Description Fixed Field
008	Fixed-Length Data Elements
020 $q	International Standard Book Number - Qualifying Information
020 $z	International Standard Book Number - Canceled/Invalid ISBN
040 $a	Cataloging Source - Original Cataloging Agency
040 $c	Cataloging Source - Transcribing Agency
043 $a	Geographic Area Code - Geographic Area Code
050 $b	(ind2=4) Library of Congress Call Number - Item Number	
082 $2	(ind1=0, ind2=4) Dewey Decimal Classification Number - Edition Number
100 $d	(ind1=4) Main Entry-Personal Name - Dates Associated with a Name
245 $c	Title Statement - Statement of Responsibility
264 $c  (ind2=4) Production, Publication, Distribution, Manufacture, and Copyright Notice - Date of production, publication, distribution, manufacture, or copyright notice
336 $a  Content Type - Content Type Term
336 $b	Content Type - Content Type Code
336 $2  Content Type - Source
337 $a  Media Type - Media Type Term
337 $b	Media Type - Media Type Code
337 $2  Media Type - Source
338 $a  Carrier Type - Carrier Type Term
338 $b	Carrier Type - Carrier Type Code
338 $2  Carrier Type - Source
504 $a	Bibliography, Etc. Note - Bibliography, Etc. Note
533 $a  Reproduction Note - Type of Reproduction
533 $b	Reproduction Note - Place of Reproduction
533 $n  Reproduction Note - Note About Reproduction
588 $a	Source of Description Note - Source of Description Note
588 $a	(ind1=0) Source of Description Note - Source of Description Note
650 $x	(ind2=0) Subject Added Entry-Topical Term - General subdivision
650 $z	(ind2=0) Subject Added Entry-Topical Term - Geographic subdivision
710 $a  (ind1=2) Added Entry-Corporate Name - Corporate name or jurisdiction name as entry element
776 $c	(ind1=0, ind2=8) Additional Physical Form Entry - Qualifying information
776 $w	(ind1=0, ind2=8) Additional Physical Form Entry - Record control number

Comment thread cds_ils/importer/providers/gobi/rules/document.py
Comment thread cds_ils/importer/providers/gobi/rules/document.py Outdated
Comment thread cds_ils/importer/providers/gobi/ignore_fields.py
@kpsherva

Copy link
Copy Markdown
Contributor

To be verified:
020__q - check if qualifier is electonik bk. if not raise an error
082__ - to check with Lydia if we need DEWEY classification
264- to check copyright notice if we need it
504 - note to be added
650 - are those subjects needed?

@callumlewis05
callumlewis05 force-pushed the new-importer-provider branch from 0d2cfad to 1093589 Compare August 31, 2026 09:21
@callumlewis05

Copy link
Copy Markdown
Contributor Author

Changes made:

  • 020 still drop field 20q but now check if electronic resource from field 007 starting with c instead
  • 082 Dewey classification is already implemented the ignored field was the edition information: "Information on the edition from which the DDC number was assigned." so kept as ignored
  • 264 Added copyright information though GOBI sample files only have copyright year
  • 504 Bibliography note not added as per Lydia's instructions but added the 500 field as a general note
  • 650 Subjects are already added as keywords but the geographic and general subdivision are kept as ignored

@callumlewis05
callumlewis05 force-pushed the new-importer-provider branch from 02e685a to 46ac85c Compare August 31, 2026 09:53
@kpsherva
kpsherva merged commit 9c069d0 into CERNDocumentServer:master Aug 31, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set up importer for new e-book provider GOBI

2 participants