I have a SpecFlow test suite running some Selenium UI tests for our web app in ADO, and we need to access some sensitive information to be able to log in to various places. They are currently stored in normal environment variables so they are in plain text and pretty easy to access, but too insecure to stay that way (it is currently a test user on a dev box, but that will change when the tests are production ready).

ADO secret env vars are not automatically set up in a way that an executable can access, and it does not look like I can pass parameters through vstest.exe to the individual tests.

I suggested something like Azure Key Vault, but that is not an option. I also got dotnet user-secrets working locally, but setting that up on all the agents is apparently not an option either.

I think that secret environment variables should be the way to go, but I am using .NET Framework 4.6.1 and all of the documentation I read to set those up with some sort of config file are for v5+ or Core. If anybody has experience dealing with this, any ideas would help!

Update: I found out it is supposedly possible to use a .runsettings file with empty parameters, and set them from within ADO, but it seems like setting the MSTest attributes conflicts with some of the SpecFlow stuff I need.

  • Can I ask why you can't use Key Vault? It's exactly the right tool for the job Jan 11 at 19:22
  • @VinceBowdren I'm not sure, I had asked my supervisor and he ruled against it. We are fully in the Microsoft ecosystem, so it really would be something good for them to set up. Jan 11 at 20:34
  • another place to securely store secrets is Azure App Configuration, but your supervisor may decide against that for the same reason as key vault, whatever that is. Jan 12 at 0:35

Ok, I think it does what I want, although I'm waiting on new db permissions to completely test it...

I found some information on using .runsettings, but I couldn't use the MSTest attributes, because the compiler only allows one initialization, which SpecFlow uses under the hood. I found SF docs to grab the context surrounding the test, but I had to annotate the class and the setter method with [Binding] and [BeforeScenario], respectively, which the docs didn't mention.

For anybody interested in the details, I created a .runsettings file in my solution directory with empty values:

    <Parameter name="api_token" value="" />
    <Parameter name="password" value="" />

And added to my .csproj file:


And in ADO, I edited the Visual Studio Test task, setting new params in Execution options > Override test run parameters:

-api_token: $(token) -password: $(password)

And wrote a C# class to get the values:

using Microsoft.VisualStudo.TestTools.UnitTesting;
using TechTalk.SpecFlow;

namespace UITests
    public class Context
        private static TestContext testContext;

        private static void SetContext(ScenarioContext scenarioContext)
            testContext = scenarioContext.ScenarioContainer.Resolve<TestContext>();

        public static string GetSecret(string key)
            if (testContext.Properties.Contains(key))
                return testContext.Properties[key].ToString();
            return null;

Using that class, the I can get the DevOps secrets from that testContext.Properties dictionary, after they overrode the empty values in my .runsettings file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.