Skip to content

Use nproc command to automatically scale to number of cores - #157

Open
olivier-thatch wants to merge 1 commit into
fly-apps:mainfrom
olivier-thatch:nproc-auto
Open

olivier-thatch wants to merge 1 commit into
fly-apps:mainfrom
olivier-thatch:nproc-auto

Conversation

@olivier-thatch

Copy link
Copy Markdown
Contributor

This PR changes the nproc option to be a boolean instead of a numeric. When the nproc option is set to true, the following line is added to the Dockerfile:

ENV MAKE="make -j$(nproc)"

which will cause the make command to automatically scale based on the number of CPU cores returned by the nproc command. nproc is part of GNU Core Utilities and should always be available.

This is a fairly common approach, e.g. https://github.com/search?q=path%3ADockerfile+%22ENV+MAKE%3D%5C%22make+-j%24%28nproc%29%5C%22%22&type=code

@luizkowalski

Copy link
Copy Markdown
Collaborator

hey olivier

when I was testing these changes, i got the following error:

> [build 2/8] RUN bundle install &&   rm -rf ~/.bundle/ "/usr/local/bundle"/ruby/*/cache "/usr/local/bundle"/ruby/*/bundler/gems/*/.git:
31.45 /usr/local/bundle/gems/bundler-4.0.18/lib/bundler/worker.rb:56:in
31.45 'Bundler::Worker#process_queue'
31.45 /usr/local/bundle/gems/bundler-4.0.18/lib/bundler/worker.rb:98:in 'block (2
31.45 levels) in Bundler::Worker#create_threads'
31.45
31.45 An error occurred while installing vernier (1.10.1), and Bundler cannot
31.45 continue.
31.45
31.45 In Gemfile:
31.45   vernier
------
Dockerfile:54
--------------------
  53 |     ENV MAKE="make -j$(nproc)"
  54 | >>> RUN bundle install && \
  55 | >>>   rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git
  56 |
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c bundle install &&   rm -rf ~/.bundle/ \"${BUNDLE_PATH}\"/ruby/*/cache \"${BUNDLE_PATH}\"/ruby/*/bundler/gems/*/.git" did not complete successfully: exit code: 5
docker stderr: Nothing written

I believe ENV can't run a shell command; it passes whatever you put into it as a string. It probably works for other people because nothing they are installing cares about make.

I believe a better approach is to use ARGS and MAKEFLAGS:

RUN MAKEFLAGS="-j$(nproc)" bundle install && ....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants