Issue with using jcvi in google colab

[whisperingkahuna]

I am using jcvi in google colab.
It installs.
I am following this tutorial.

Download data

I run this command

!python -m jcvi.apps.fetch phytozome

and it appears to be working fine

I get the following output.

--2024-01-02 17:53:38--  http://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12
Resolving genome.jgi.doe.gov (genome.jgi.doe.gov)... 104.18.38.140, 172.64.149.116, 2606:4700:4400::ac40:9574, ...
Connecting to genome.jgi.doe.gov (genome.jgi.doe.gov)|104.18.38.140|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12 [following]
--2024-01-02 17:53:38--  https://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12
Connecting to genome.jgi.doe.gov (genome.jgi.doe.gov)|104.18.38.140|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://genome.jgi.doe.gov/portal/ext-api/downloads/get-directory?organism=PhytozomeV12 [following]
--2024-01-02 17:53:38--  https://genome.jgi.doe.gov/portal/ext-api/downloads/get-directory?organism=PhytozomeV12
Reusing existing connection to genome.jgi.doe.gov:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: '.phytozome_directory_V12.xml'

.phytozome_director     [   <=>              ] 480.05K   699KB/s    in 0.7s    

2024-01-02 17:53:40 (699 KB/s) - '.phytozome_directory_V12.xml' saved [491576]

Usage: 
    fetch.py phytozome species

    Retrieve genomes and annotations from phytozome using Globus API. Available
    species listed below. Use comma to give a list of species to download. For
    example:

    $ fetch.py phytozome Athaliana,Vvinifera,Osativa,Sbicolor,Slycopersicum

    The downloader will prompt you to enter Phytozome user name and password
    during downloading. Please register for a login at:
    https://phytozome.jgi.doe.gov/pz/portal.html.
    
                 ZmaysPH207                      Zmays
                    Zmarina                   Vcarteri
                  Tpratense                   Sviridis
                 Stuberosum                  Spurpurea
            Smoellendorffii              Slycopersicum
                    Sfallax                   Sbicolor
                  Pvulgaris                   Ppersica
                  Othomaeum             OsativaKitaake
                    Osativa                Mtruncatula
                  MspRCC299                Mpolymorpha
                 Mesculenta                 Macuminata
                 Klaxiflora              Kfedtschenkoi
                     Fvesca                   Egrandis
                    Dsalina                    Dcarota
        CsubellipsoideaC169                    Cpapaya
          Boleraceacapitata                Bdistachyon
                    Alyrata           Ahypochondriacus
                   Acomosus                  Acoerulea
      Vunguiculata_469_v1.1      Tintermedium_503_v2.1
         Taestivum_296_v2.2          Sviridis_500_v2.1
       SbicolorRio_468_v2.1         Pvirgatum_450_v4.1
      Pumbilicalis_456_v1.5      Ptrichocarpa_444_v3.1
           Phallii_495_v3.1        PhalliiHAL_496_v2.1
    PdeltoidesWV94_445_v2.1         Oeuropaea_451_v1.0
         Msinensis_497_v7.1             Lsativa_467_v5
            Hvulgare_462_r1           Hannuus_494_r1.2
         Ghirsutum_458_v1.1 Czofingiensis_461_v5.2.3.2
           Cquinoa_392_v1.0        Carietinum_492_v1.0
       Bsylvaticum_490_v1.1         Bhybridum_463_v1.1
 BdistachyonBd21_3_460_v1.1          Bbraunii_502_v2.1
    Athaliana_447_Araport11      Aofficinalis_498_V1.1
      Aoccidentale_449_v0.9  Ahypochondriacus_459_v2.1

Options:
  -h, --help            Show this help message and exit
  --version=VERSION     Phytozome version, must be one of 9|10|11|12|12_unrestricted|13 [default:
                        12]
  --assembly            Download assembly [default: False]
  --format              Format to CDS and BED for synteny inference [default: False]
  --downloader=DOWNLOADER
                        Use the specified downloader to retrieve resources, must be one of
                        curl|insecure|powershell|wget [default: guess]

JCVI utility libraries 1.3.9 [Copyright (c) 2010-2024, Haibao Tang]

I now run this command
!python -m jcvi.apps.fetch phytozome Vvinifera,Ppersica

I get this response

[01/02/24 17:57:20] INFO File .phytozome_directory_V12.xml exists. Download
I upload the file for reference.
phytozome_directory_V12.zip

I delete this file from the workspace and ran the below command again
!python -m jcvi.apps.fetch phytozome Vvinifera,Ppersica

I get this response

--2024-01-02 18:07:56--  http://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12
Resolving genome.jgi.doe.gov (genome.jgi.doe.gov)... 172.64.149.116, 104.18.38.140, 2606:4700:4400::ac40:9574, ...
Connecting to genome.jgi.doe.gov (genome.jgi.doe.gov)|172.64.149.116|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12 [following]
--2024-01-02 18:07:56--  https://genome.jgi.doe.gov/ext-api/downloads/get-directory?organism=PhytozomeV12
Connecting to genome.jgi.doe.gov (genome.jgi.doe.gov)|172.64.149.116|:443... connected.
HTTP request sent, awaiting response... 307 Temporary Redirect
Location: https://genome.jgi.doe.gov/portal/ext-api/downloads/get-directory?organism=PhytozomeV12 [following]
--2024-01-02 18:07:57--  https://genome.jgi.doe.gov/portal/ext-api/downloads/get-directory?organism=PhytozomeV12
Reusing existing connection to genome.jgi.doe.gov:443.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: '.phytozome_directory_V12.xml'

.phytozome_director     [   <=>              ] 480.05K   708KB/s    in 0.7s    

2024-01-02 18:08:00 (708 KB/s) - '.phytozome_directory_V12.xml' saved [491576]

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/site-packages/jcvi/apps/fetch.py", line 727, in <module>
    main()
  File "/usr/local/lib/python3.10/site-packages/jcvi/apps/fetch.py", line 137, in main
    p.dispatch(globals())
  File "/usr/local/lib/python3.10/site-packages/jcvi/apps/base.py", line 140, in dispatch
    globals[action](sys.argv[2:])
  File "/usr/local/lib/python3.10/site-packages/jcvi/apps/fetch.py", line 293, in phytozome
    res = download_species_phytozome(
  File "/usr/local/lib/python3.10/site-packages/jcvi/apps/fetch.py", line 325, in download_species_phytozome
    assert species in valid_species, "{} is not in the species list".format(species)
AssertionError: Vvinifera is not in the species list

I don't know what the problem is.
PS: I do not have a background in bioinformatics; I am helping someone execute the code. I would appreciate any lead

[YingChen94]

I also had the same issue. Just use a different species, such as banana "Macuminata" python -m jcvi.apps.fetch phytozome Macuminata

The seqids will be different:

chr1,chr2,chr3,chr4,chr5,chr6,chr7,chr8,chr9,chr10,chr11
Pp01,Pp02,Pp03,Pp04,Pp05,Pp06,Pp07,Pp08

And I got the plot:

[tanghaibao]

@whisperingkahuna

This is due to that Phytozome removed access to Vvinifera (grape) for some reason. I'll have to use a different organism for the wiki tutorial. Thanks for reporting this issue.

Haibao

[tanghaibao]

The wiki is temporarily upgraded to reflect the unavailability of grape via direct download. Please use the portal for a manual download.