Hackpads are smart collaborative documents. Join Hackpad Now.
1 day ago
Unfiled. Edited by Phil Elson 1 day ago
Phil E Thoughts on verification:
  • We could emit a SHA within the CI log, propose an upload, and get the service to cross validate that the tarball being uploaded matches the SHA of the build.
 
 
Reminder: remove this document once we have formulated an enhancement proposal
 
2 days ago
16 / 26
Phil E
  • 2016-??-??: General discussion
Time: 14:00 UTC
 
John K Attendees
Phil E Standing items
  • How many repos?
  • How many contributors?
  • New core devs? 
 
 
 
Phil E
  • Feedstocks philosophy:  Explicit vs implicit / reproducible vs redundant
  • OSX - getting back to a usable, coherent, stack
msarahan@continuum.io
  • libc++ (clang) vs libstdc++ (gcc/g++)
  • Minimum OSX required for clang (10.8, I think?)
John K
  • Actually clang is usable beginning in 10.7. So, this would be viable given your compatibility constraints.
  • Also, all the refs I have seen suggest that this will still have C++11 support.
msarahan@continuum.io
  • Compatibility with defaults (built on 10.7, uses gcc) - where will people break?  I think only if mixing packages - how do we assure that we have all the ones we need?
  • Metadata unification with Continuum - are we OK with adding some fields to about section to match Anaconda standard?
  • license_family
  • doc_url
  • dev_url
  • constrain summary to 80 chars (longer stuff use description)
  • Can we add this to linter, and add to recipes as we update them?
  • What support for unicode should we have?  Any?  Summary/description only?
John K
  • CUDA/cuDNN update
  • Improving infrastructure
  • Better workflows with staged-recipes
  • Notifications (how do we stay on top of them)
  • MSYS2
msarahan@continuum.io
  • Available on defaults - was in conda 4.1.7, but that was pulled.  Coming in 4.1.8.
John K
  • Discussing Ray Donnelly's work on MSYS2 packages and how we want to use and integrate these into conda-forge.
  • Some use cases to consider OpenBLAS, FFTW, build tools, others?
  • Binary data
  • Do we include it in recipes?
  • What kinds do we allow if any (e.g. icons)?
  • How do we verify the licensing?
  • How do we verify that they are safe?
  • Dev releases: Where do they happen?
  • Do we do them at conda-forge?
  • Maybe add a label.
  • Do we let others do them with a feedstock on their own repo?
  • How do we enforce whatever we decide?
  • Channel mirroring
Björn G
  • Can this point be a little bit explained? I thought about this as well and would like to contribute to this point.
Matt C
  • conda-build-all can copy from one channel to another: `conda build-all --inspect-channels conda-forge --upload-channels astropy some_packge_recipe` will copy the `some_package` from the channel conda-forge to  astropy if it can, or build it if it doesn't exist on conda-forge. Discussion  about what the desired behavior should be has started at: SciTools/conda build all#46
John K
  • Feedstock history
  • Is it sacred?
  • Do we rebase/force push?
  • If so, under what conditions?
  • How do we avoid multiple people doing this simultaneously?
msarahan@continuum.io
  • I don't think you can.
  • IMHO, if it's just one author in staged recipes, sure.  If feedstock, no force push - only to PRs to feedstock.  If people don't mind merge PRs, it sure is a lot simpler to not rebase.  I have messed up rebasing a few times recently... =(
ocefpaf@gmail.com
  • Drop numpy 1.10 and reduce our build matrix. (Numba now works with numpy 1.11.)
John K
  • Signing packages
  • There has been some interest previously.
  • HTTPError: 503 Server Error: Service Unavailable: Back-end server is at capacity for url...
  • Seems we are regularly running into this issue under normal usage conditions.
  • Had discussed previously caching packages on AppVeyor and trying to reuse those to start.
  • Maybe we need to consider caching on all CIs.
  • Building our own Miniconda-like self-extracting scripts with packages via `constructor`.
msarahan@continuum.io
  • There have been improvements on Continuum's side that should help this.  In short, repodata (the package index for a given channel) was being generated for each anaconda.org query.  This was unnecessarily high cost, and some caching schemes have been implemented.
John K
  • Handling removal of unpinned/improperly pinned packages.
  • Has been done manually thus far.
  • This doesn't scale well though.
  • Should we (semi) automate removal?
  • Should we label them as broken
  • Not currently buildable packages
  • In particular open source code that is out of scope for CIs.
  • Examples include Qt4, Qt5, possibly PyQt4, possibly PyQt5, gcc, VTK, etc. 
Ray D
  • Continuum Qt5 and PyQt5 builds coming soon™
John K
  • How do we indicate they are built manually?
  • Are we ok with uploading non-built binaries?
  • When do we determine something is ok to be built manually?
  • What procedures should people follow for building manually?
msarahan@continuum.io
  • Use a standard build docker image, VM, or vagrant file
  • Sign package?
  • Implement reproducible builds where feasible (linux)
John K
  • What changes do we need to make in conda-smithy elsewhere?
  • What other build infrastructure could we utilize?
msarahan@continuum.io
  • Would be nice to provide some volunteer builder abstraction, so that we could have an elastic worker farm that would be somewhat resilient.
  • Standardizing build images is probably (relatively) easy - how to orchestrate, though?
...
7 days ago
Unfiled. Edited by John Kirkham 7 days ago
John K
 
98 days ago
Unfiled. Edited by John Kirkham 98 days ago
  • Build number should start at `(X+1)*100` - Phil: why? It shouldn't make a difference with the "feature" formed numpy that we have built....? - John: See this comment for response.
 
Members (23)
Ray Donnelly thedizzle@gmail.com Jake Vanderplas Björn Grüning jjhelmus@gmail.com Mark Mikofski Amir Mohammadi Matt Craig Carl Kleffner Johannes Köster Mario Juric Stuart Berg Jan Schulz ocefpaf@gmail.com scopatz@gmail.com thomas@kluyver.me.uk Carol Willing Michael Sarahan Patrick Snape edill@bnl.gov
Collections

Create a New Collection

Cancel

Move XXX to XXX


XXX will be invited to the XXX on XXX.

Cancel

Contact Support



Please check out our How-to Guide and FAQ first to see if your question is already answered! :)

If you have a feature request, please add it to this pad. Thanks!


Log in / Sign up