News

Backend Migration of Neotoma DB, new API release

For the last several years, the Neotoma IT team (Eric Grimm, Simon Goring, Mike Stryker, and Steve Crawford; special thanks to Anna George, Jack Williams, and Jessica Blois) has been migrating the Neotoma relational database from SQL Server (Windows-based) to PostgreSQL (Linux). Most of the effort associated with this migration has gone into transferring and testing all Tilia Data Steward services. The goal of this migration is to make Neotoma more sustainable and easily supported by a network of open-source developers.

All migration work so far has been done on a development version of the database, leaving the mainline production version (i.e. the one used by most people) unchanged.  As the last stage of the migration process, we are now switching over the production version of the database.  This process began Aug 10 and we hope to have the first wave of migration done by Friday Aug 21.  Once this migration is complete, there will be two versions of the database:  a primary living version in PostgreSQL, and a secondary static version, maintained for backwards compatibility, on SQL Server.  The migration is anticipated to affect services as follows:

Explorer:  Explorer services have been migrated to the new server and tested.  There may be a few temporary disruptions, with a known bug to be fixed for Stratigraphic Diagrammer.

Tilia:  Tilia has been migrated and tested against the PostgreSQL server and the updated API (currently at http://tilia-dev.neotomadb.org).  Some Tilia-server interactions may be slower than in the past as we optimize the Postgres database, but should return to normal shortly.

API:  There are new generations of APIs (v1.5, v2.0) that point to and work with the PostgreSQL version of the database, available temporarily here: http://api-dev.neotomadb.org/api-docs/ and a soon-to-be-established long-term URL here: http://api.neotomadb.org/api-docs/. The original APIs (v1.0) still point to the static SQLServer/Windows database.  These V1.0 APIs are being renamed with a ‘wn’ prefix added, e.g. https://wnapi.neotomadb.org/v1/data/sites?sitename=Marion%  

neotoma R package: The original neotoma R package is built on top of v1.0 APIs, and so points to the now-static SQLServer version of the database. The source code of the R package has been pointed to the new Windows server.  These changes have been pushed to GitHub (http://github.com/ropensci/neotoma) and we are awaiting a full release on CRAN. A new Neotoma R package (neotoma2, breaking compatibility with the old package) will point to the living PostgreSQL version.  V2 is expected to be released in late October, with pre-release expected soon.  Development for this package can be found here: http://github.com/NeotomaDB/neotoma2

DOIs:  As a newer feature, DOIs were designed to work with the PostgreSQL version. There may be a few bugs or temporary disruptions to DOI minting and accessibility through landing pages, but we anticipate that these will be brief.

Science article: Human effects on ecosystems

Hi all,
Nice reporting by Science magazine on a talk presented at the 2020 Ecological Society of America meeting, by Allison Stegner and Trisha Spanbauer, showing evidence from Neotoma indicating that the impacts by human land use on ecosystems was as large or larger than the effects associated with the end of the last deglaciation https://www.sciencemag.org/news/2020/08/humans-have-altered-north-america-s-ecosystems-more-melting-glaciers

Posted by Jack Williams on 08/13

Server upgrade July 1, temporary closure of Neotoma services

Dear Colleagues,

Beginning Wednesday, July 1, Neotoma will be down for several days during a server upgrade at Penn State University.  This downtime will affect the Neotoma database and all software services associated with it (e.g. Neotoma Explorer, APIs, R package, Tilia-to-Neotoma services).  We do not yet know the exact duration of this downtime but hope it will be less than a week.  All Neotoma services on the new server are being tested during this upgrade, to ensure that everything runs smoothly afterwards.  However, there is a possibility that some Neotoma services will need patching after this server upgrade. 

We apologize for the short advance notice of this upgrade and for any disruptions caused.  We are monitoring this situation closely and will post updates as soon as we have them.

-Jack Williams, Jessica Blois, Steve Crawford, Simon Goring, Eric Grimm, Doug Miller, Mike Stryker

PS Many of you know that we are also working on a backend migration of the Neotoma database from a SQL Server/Windows server to a PostgreSQL/Linux server.  That migration is nearly complete but is separate from this PSU server upgrade.  We likely will need to schedule a second downtime window to complete this migration.  We will send out updates as soon as we establish the timeline for this database migration.

[Update 6/26/2020 3:13pm US CT: changed contact email to Neotoma-contact@googlegroups.com]

[Update 8/5/2020 The server upgrade is underway and we are experiencing a few broken links as we repoint services from the old PSU server to the new one. Additionally, services are running slower than normal, which we are working on as well. We are working to get these issues resolved as fast as possible.]

New bylaws and ECRs approved!

The Leadership Council approved the new bylaws adding early-career representatives to the council as full voting members!  9 people voted, 9 yes to 0 no.  In addition, after internal discussion the LC unanimously voted to elevate the ad hoc ECR reps to full voting reps rather than holding a new election, since they were just voted into their roles in late fall 2019.  So welcome to Dana Reuter and Suzette Flantua to the Neotoma Leadership Council!  They will both be on the council through the end of their original term, December 2021.  

Posted by Jessica Blois on 05/16

New constituent database added to Neotoma!

The Leadership Council unanimously approved the addition of a new constituent database to the Neotoma Paleoecology Database! The new database is named PaVeLA (PaleoVertebrates of Latin America). Joaquín Arroyo-Cabrales (Instituto Nacional de Antropología e Historia, México) is the lead for the database, and is working with graduate student Deborah V. Espinosa-Martínez and overall vertebrates lead Jessica Blois on integrating the Quaternary Mammals of Mexico Database into PaVeLA.  If you are interested in adding sites to the database, or learning more, please contact Joaquín Arroyo-Cabrales (arromatu@hotmail.com) or Jessica Blois (jblois@ucmerced.edu).

Posted by Jessica Blois on 04/22

Ancient Environmental DNA Cyberinfrastructure Workshop: Applications Open

Dear colleagues,

Neotoma and the University of Wisconsin will be hosting a workshop to lay the groundwork for a robust cyberinfrastructure dedicated to supporting the open access for ancient environmental DNA and integrating it with existing cyberinfrastructure in genomics and paleoecology.  This workshop will be held May 18-20 in Madison, WI, USA.  The workshop is supported by the National Science Foundation and standard travel costs will be covered for all participants.

This ca. 20-person workshop includes several open participant slots specifically dedicated to early career scientists and scientists from underrepresented populations.  If you are interested in attending, please email your CV and a short statement of research interests to Jack Williams (jwwilliams1@wisc.edu) by March 6.

Planned activities include: 1) review the current state of the art with respect to ancient DNA data generation, data handling, data archival, and analytical pipelines; 2) review the latest advances and capabilities of existing cyberinfrastructures in genomics and paleoecology; 3) identify paleogenomic-relevant gaps and semantic misalignments among existing resources; 4) establish priorities and initial standards for data and metadata reporting in community paleodata resources. The Steering Committee for this workshop is Inger Alsos, Jessica Blois, Mary Edwards, Eric Grimm, Laura Parducci, Beth Shapiro, and Jack Williams. 

If you have any questions please reach out to Jack Williams. 
 

Posted by Jack Williams on 02/17

Data Use Policy Updated

The Neotoma Data Use Policy has been updated and posted.  Neotoma data are provided under a CC-BY 4.0 license and are open for use with citation.  The revised Data Use Policy clarifies that proper citation of Neotoma data operates at three levels:  Neotoma itself, the Constituent Database(s) that curate these data, and the original data providers.  Appendix 1 provides standard citation endpoint and acknowledgment text for Neotoma itself, while Appendix 2 provides preferred citation endpoints and acknowledgments for individual Constituent Databases.

Posted by Jack Williams on 01/15

Quaternary Times 2019 Newsletter

For an update on recent Neotoma activities, check out the article published in the fall 2019 issue of the Quaternary Times, the newsletter for the American Quaternary Association (AmQua).  https://www.amqua.org/public/news/Quaternary Times_Vol. 41 Fall 2019-2.pdf

Posted by Jack Williams on 11/19