Skip to main content
DevOps

Return to Answer

added 48 characters in body
Source Link
Ta Mu
Ta Mu
  • 6.8k
  • 5
  • 42
  • 83

TLDR; set workers' memory to 4G.

I reduced the test to the build-in Pi calculation:

./bin/run-example SparkPi 10

And got the same result; apparently 1G memory is insufficient even for simplest experiments. Increasing both master and workers memory to 4G and the problematic behaviour was not reproducible. However before the systems start computation the message still could appear but just once, seems it is printed in a waiting loop.

I reduced the test to the build-in Pi calculation:

./bin/run-example SparkPi 10

And got the same result; apparently 1G memory is insufficient even for simplest experiments. Increasing both master and workers memory to 4G and the behaviour was not reproducible. However before the systems start computation the message still could appear but just once, seems it is printed in a waiting loop.

TLDR; set workers' memory to 4G.

I reduced the test to the build-in Pi calculation:

./bin/run-example SparkPi 10

And got the same result; apparently 1G memory is insufficient even for simplest experiments. Increasing both master and workers memory to 4G and the problematic behaviour was not reproducible. However before the systems start computation the message still could appear but just once, seems it is printed in a waiting loop.

Source Link
Ta Mu
Ta Mu
  • 6.8k
  • 5
  • 42
  • 83

I reduced the test to the build-in Pi calculation:

./bin/run-example SparkPi 10

And got the same result; apparently 1G memory is insufficient even for simplest experiments. Increasing both master and workers memory to 4G and the behaviour was not reproducible. However before the systems start computation the message still could appear but just once, seems it is printed in a waiting loop.