R CRAN package migration.

[trivialfis]

We will try to submit the new CRAN release before the end of November.

Following the 3.0.0 release (now 3.1), we published the latest R package on the R universe. We would love to bring the latest package back to the CRAN repository. Starting this thread to help track any issues during migration.

• R package document: https://xgboost.readthedocs.io/en/stable/R-package/index.html
• Migration guide: https://xgboost.readthedocs.io/en/stable/R-package/migration_guide.html
• API reference: https://xgboost.readthedocs.io/en/stable/r_docs/R-package/docs/reference/index.html
• R universe: https://dmlc.r-universe.dev/xgboost

To install the latest XGBoost from r-universe:

install.packages('xgboost', repos = c('https://dmlc.r-universe.dev', 'https://cloud.r-project.org'))

References:

• Discussion thread for development.
• Roadmap thread.
• Original change proposal.

rev-check

• Results from tools::check_packages_in_dir:
  errpkgs.zip
  I have split up the results for each package. Please download the zip file and find the name of your package.
• Results from revdepcheck::revdep_check:
  revdepcheck-results.zip
  The revdepcheck is not particularly useful. The check did not go smoothly, just including it here as an extra reference.

documentation
Use the "stable" tag now that 3.1.0 is released.
Please use the "latest" readthedocs links until 3.1.0 is formally released.

Questions

• My package is not in the list, but I received the email.
  I cc'd the R development list in the notification email. If your email address is not in the To list, please ignore the email. I'm sorry for the noise.

[trivialfis]

I have sent the update email to downstream project maintainers. cc @david-cortes Aside from instantly being tagged as [Possible SPAM/PHISHING] for some recipients, we will see how it goes.

[bbolker]

Looking at the zip file, I don't see the names of any packages that I maintain. I might have missed something (I didn't do a formal check), but is there any chance you could double-check to see if my name was included in the e-mail blast by accident?

[deweyme]

Like bbolker I do not see any of my packages in the zip file so I suspect I am a false positive.

[trivialfis]

Apologies for the confusion, if you don't see your package in the results, then it's likely that your package is failing the check due to missing dependencies in my environment. I looked up some of those failures, they are caused by archived CRAN packages. If you don't consider it an issue, please ignore my email.

Update: see the comment below.

[trivialfis]

I obtained the list of emails through the following snippet:

library(memoise)
library(stringr)
library(tools)
library(fs)

details <- check_packages_in_dir_details(
  "./r-revs/r-reverse-4/deps"
)
summary(details)

cat("\n")

errors <- details[details$Status == "ERROR", ]
summary(errors)

outdir <- "errpkgs"

cm <- cachem::cache_disk("./cache")

cran_packages <- memoise::memoise(
  function() {
    local <- path_temp("packages.rds")
    utils::download.file("https://cran.R-project.org/web/packages/packages.rds", local,
      mode = "wb", quiet = TRUE
    )
    on.exit(file_delete(local))
    cp <- readRDS(local)
    rownames(cp) <- unname(cp[, 1])
    cp
  },
  cache = cm
)

cran <- cran_packages()

for (i in seq_len(nrow(errors))) {
  pkg <- errors[i, ]
  cre <- cran[pkg$Package, "Maintainer"]
  name <- name <- str_extract(cre, ".*?(?=\\s*<)")
  email <- str_extract(cre, "(?<=<).*?(?=>)")

  outstr <- capture.output(print(pkg))
  if (grepl("Packages required but not available", paste(outstr, collapse = "\n"))) {
    next
  }
  pkgpath <- file.path(outdir, pkg$Package)
  cat(email, "\n")
  if (file.exists(pkgpath)) {
    unlink(pkgpath)
  }
  writeLines(outstr, pkgpath)
}

[trivialfis]

Hi, I cc'd the r-package-devel@r-project.org for the notification email. If your email address is not in the To field of the email, then it's likely that you received the email due to subscribing to the R mailing list. Apologies for the noise, I used the CC to get attention from maintainers in case if there's a better solution.

[bastistician]

We will try to submit the new CRAN release before the end of November.

Checking the xgboost 3.1.1.1 source package against CRAN as of yesterday:

Check status summary:
                  ERROR  OK
  Source packages     0   1
  Reverse depends    55 114

Check                              ERROR
  examples                            28
  re-building of vignette outputs     20
  tests                               29
  whether package can be installed     2

Details per package: ERROR.zip

So, unfortunately, revdep health has not improved much. If not done already, I think it would be good to send reminders to the maintainers of CRAN packages that are still not compatible with the new 'xgboost' interface. Ideally packages should maintain compatibility with both the old and new interfaces, and release an update in advance to minimize repository disruption. A clearly communicated release schedule should help push maintainers.

[trivialfis]

@bastistician Thank you for sharing the results! I will send a reminder and see if I can get some feedback.

[bastistician]

Note that 2 CRAN packages ('personalized', 'EIX') fail to be installed with the new 'xgboost' version, in both cases due to

Error: object 'slice' is not exported by 'namespace:xgboost'

To avoid such severe breakage, it would be better to keep exporting a slice function from 'xgboost', presumably one that just calls .Defunct(msg = "<something informative>").

More generally: Is there advice for package authors on how to retain simultaneous compatibility with the old and new interfaces? The changes documented in the migration guide look quite significant. I expect that in some cases package authors will choose to declare xgboost (>= 3) (or whatever minimum is required) and release their adjusted package just after the new 'xgboost' interface is released.

[trivialfis]

Sent a reminder email.

The changes documented in the migration guide look quite significant.

Indeed. The R package was written years ago, it was a very different world.

To avoid such severe breakage, it would be better to keep exporting a slice function from 'xgboost', presumably one that just calls .Defunct(msg = "").

I will ping @david-cortes for opinions.

[david-cortes]

I don't think it'd be a good idea to bring back the slice method. It's not for nothing that it was removed (conflicts with dplyr's). Guess those packages could add conditional statements on version or requireNamespace to be compatible with both the old and new releases, or wait until the next release to make their changes.