Improve hosted build agent performance with build caches
This suggestion is migrated to Developer Community. Please use below link to view the current status.
Consider options to reduce build time of hosted build agent significantly. I don't have problem paying for improved performance if there were such option.
For example dotnet restore build task takes 95 seconds for simple asp.net core 2 app in each build. Packages could be cached.
Agent initialization takes 39 seconds. Agent could be initialized in advance, why not?
We’re actively working on this now but I’ll make sure it is being tracked in the right places internally before I mark it officially Planned.
Mike Brosnan commented
There are many features in Artifactory around artefact and dependency management, are you working on a MVP for dependency management/ proxying maven repos, what is the roadmap?
Andrew Stoker commented
Adding my voice here. Building locally (when we have a cache) takes about 30 seconds. Without the cache, we're downloading a lot and our build times take upwards of 5 minutes. Having a cache is extremely important to our build process.
Any update on this? Downloading a 2.5GB SDK we need every time we build really slows our builds.
Dennis Hsu commented
Can we update the NodeJS version on hosted agent?
I used Node.JS Tool installer to install LTS 8.12.0 and it reduced our NPM install step by 32 seconds, but the install of 8.12.0 took 39 seconds, so no overall gain.
But can Azure DevOps install the latest Node.JS module on hosted agent? or allow user select which version of NodeJS to use?
AJ Soto commented
Does anyone know if by using Azure Artifacts - and setting up a feed that allows packages from public sources (like nuget), will make the builds from a hosted agent any faster since if would be getting the packages from the Azure Artifacts feed (after of course the initial download/restore of all the nuget packages) instead of from NuGet?
Frantisek Skorunka commented
Any update on this?
Travis has support for build caches and it dramatically speeds up tests.
Aaron Aichlmayr commented
Any news on this one? Even if this would be an azure disk resource we could have mounted it would solve the need
@Admin, @VSTS Team:
would you please update?
Is it already "being tracked in the right places internally"?
Also, I'm very curious, what could be causing @Michael Bruyninckx's problem: "Every NPM interaction (like "npm --version" or "npm set progress=false") takes about 3 minutes. Every NPM step we add, adds 15 minutes extra buildtime". It does not seem like cache problem
Michael Yermian commented
Let's keep in mind that packages could potentially conflict...
Company A has a private feed with a package named 'Utilities.1.0.0.nupkg' and Company B has a different package with the same identifier.
If anything is cached it should only cache very common sources, nuget.org, npmjs.com, etc.
Technically, a project could target only a private feed which has a conflicting package identifier... case in point, Company A has their own package named 'Utlities.2.0.0.nupkg' in their private feed, but there is already a package named Utilities in nuget.org (https://www.nuget.org/packages/Utilities/2.0.0)... how would the build agent decide if it should use your private feed or the nuget.org cache version of Utilities.2.0.0?
Michael Bruyninckx commented
Every NPM interaction (like "npm --version" or "npm set progress=false") takes about 3 minutes. Every NPM step we add, adds 15 minutes extra buildtime !!! that's unacceptable !
Laurence Mee commented
Just as a FYI. Creating a private NuGet feed which also pulls from nuget.org did speed up things quite a bit - 5 minutes instead of 25.
Laurence Mee commented
My Nuget restore in a VSTS build is also taking far too long. .NetCore2 builds typically 25 minutes for a nuget restore, converted it to NetCore2.1 and it is now over 60 minutes so the build fails.
Frantisek Skorunka commented
This is a must have. Easy to implement and huge benefit for users.
Amir Mechouk commented
Can we get an update to this? Last official update was back in February.
Caching docker images would be invaluable. 90% of my build time is waiting for images.
Tarun Babbar commented
Do we have any updates on this? How to improve build time on hosted agents?
Michel Zehnder commented
And here I thought "the Cloud" is fast and scalable. Seems I was wrong :D
Just want to stress that ALL tasks are very slow in VSTS. Not only npm install or NuGet restore, but, for example, npm build is also VERY slow. Takes 30 seconds to build an Angular app on my average laptop at 30% CPU, but takes 3 minutes in VSTS! Seems that there are few VSTS agents working a lot of jobs, or the agents themselves are very weak. I work in a huge company, I'm sure they'll gladly pay you money to have more juice in this thing, as long it's proportional money. BTW npm install I've improved a bit by installing npm 5.8 each time and running the new "npm ci" feature that uses package-lock.json.
Leonardo Galli commented
Please increase the priority of this item! Every other CI / CD tool has this feature (CircleCI, appveyor, etc.) This would not only be useful for dotnet restore, but also git clone, npm install and other custom build / package installation tools.