REQUEST: Separate performance specs from other specs
-
macournoyer
- Milestone set to 0.8.0
- State changed from new to open
Sure all specs pass on my machine, I'm a big TDD/BDD fan ;)
benchmark_unit is supposed to be machine independent but I guess it's not 100% correct.
I like the idea of isolating the perf specs, I'll try to do that.
Can you paste the output of your failures?
-
Wincent Colaiuta
Yes; here are the timing-related failures:
2) 'Thin::Response should be fast' FAILED took <0.000141749723145072 RubySeconds>, should take less than 0.00011 RubySeconds. /home/wcolaiuta/work/thin/thin.git/spec/response_spec.rb:63: 11) 'Thin::Server on TCP socket should handle GET in less then 0.0045 RubySecond' FAILED took <0.00973499188750674 RubySeconds>, should take less than 0.0045 RubySeconds. /home/wcolaiuta/work/thin/thin.git/spec/server/tcp_spec.rb:40: 12) 'Thin::Server on TCP socket should handle POST in less then 0.007 RubySecond' FAILED took <0.008859357696567 RubySeconds>, should take less than 0.007 RubySeconds. /home/wcolaiuta/work/thin/thin.git/spec/server/tcp_spec.rb:44: 16) 'Thin::Request performance should be faster then 0.0002 RubySeconds' FAILED took <0.000255813953488372 RubySeconds>, should take less than 0.0002 RubySeconds. /home/wcolaiuta/work/thin/thin.git/spec/request/perf_spec.rb:21:And here are the other failing cases (haven't looked into them very closely yet):
1) 'Thin::Connection should return remote_address' FAILED expected: "127.0.0.1", got: nil (using ==) /home/wcolaiuta/work/thin/thin.git/spec/connection_spec.rb:55: 3) NoMethodError in 'Thin::Logging should output debug when set to true' protected method `debug' called for #<TestLogging:0xb7c7cfec> /home/wcolaiuta/work/thin/thin.git/spec/logging_spec.rb:16: 4) NoMethodError in 'Thin::Logging should output trace when set to true' protected method `trace' called for #<TestLogging:0xb7c7b4e4> /home/wcolaiuta/work/thin/thin.git/spec/logging_spec.rb:22: 5) NoMethodError in 'Thin::Logging should not output when silenced' protected method `log' called for #<TestLogging:0xb7c79810> /home/wcolaiuta/work/thin/thin.git/spec/logging_spec.rb:28: 6) NoMethodError in 'Thin::Logging should not output when silenced [deprecated]' protected method `log' called for #<TestLogging:0xb7c78078> /home/wcolaiuta/work/thin/thin.git/spec/logging_spec.rb:36: 7) 'Daemonizing should send kill signal if timeout' FAILED expected false, got true /home/wcolaiuta/work/thin/thin.git/spec/daemonizing_spec.rb:112: 8) NoMethodError in 'Thin::Server app builder should work with Rack url mapper' You have a nil object when you didn't expect it! You might have expected an instance of Array. The error occurred while evaluating nil.+ /home/wcolaiuta/work/thin/thin.git/spec/server/builder_spec.rb:36: 9) 'Thin::Server on UNIX domain socket should retreive remote address' FAILED expected "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-Length: 519\r\nConnection: close\r\nServer: thin 0.7.0 codename Spherical Cow\r\n\r\n{\"rack.url_scheme\"=>\"http\", \"PATH_INFO\"=>\"/\", \"rack.run_once\"=>false, \"rack.input\"=>#<StringIO:0xb7df6e04>, \"SCRIPT_NAME\"=>\"\", \"SERVER_PROTOCOL\"=>\"HTTP/1.1\", \"rack.errors\"=>#<IO:0xb7ef501c>, \"REMOTE_ADDR\"=>\"V\", \"REQUEST_PATH\"=>\"/\", \"SERVER_SOFTWARE\"=>\"thin 0.7.0 codename Spherical Cow\", \"HTTP_VERSION\"=>\"HTTP/1.1\", \"rack.multithread\"=>false, \"rack.version\"=>[0, 2], \"REQUEST_URI\"=>\"/\", \"rack.multiprocess\"=>false, \"QUERY_STRING\"=>\"\", \"GATEWAY_INTERFACE\"=>\"CGI/1.2\", \"HTTP_CONNECTION\"=>\"close\", \"REQUEST_METHOD\"=>\"GET\"}" to include "\"REMOTE_ADDR\"=>\"\"" /home/wcolaiuta/work/thin/thin.git/spec/server/unix_socket_spec.rb:15: 10) Errno::ENOENT in 'Thin::Server on UNIX domain socket should handle GET in less then 0.002 RubySecond' No such file or directory - /tmp/thin_test.sock /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:154:in `initialize' /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:154:in `new' /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:154:in `send_data' /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:166:in `get' /home/wcolaiuta/work/thin/thin.git/spec/server/unix_socket_spec.rb:19: /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:29:in `times' /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:29:in `matches?' /home/wcolaiuta/work/thin/thin.git/spec/controllers/../spec_helper.rb:28:in `matches?' /home/wcolaiuta/work/thin/thin.git/spec/server/unix_socket_spec.rb:19: 13) Errno::ECONNREFUSED in 'Thin::Server on Swiftiply should GET from Net::HTTP' Connection refused - connect(2) /home/wcolaiuta/work/thin/thin.git/spec/server/swiftiply_spec.rb:16: 14) Errno::ECONNREFUSED in 'Thin::Server on Swiftiply should POST from Net::HTTP' Connection refused - connect(2) /home/wcolaiuta/work/thin/thin.git/spec/server/swiftiply_spec.rb:20: 15) 'Thin::UnixConnection should return remote_address' FAILED expected: "127.0.0.1", got: nil (using ==) /home/wcolaiuta/work/thin/thin.git/spec/connectors/unix_server_spec.rb:41: 17) 'Thin::Request legacy Mongrel tests should raise error on large mangled field values' FAILED expected Thin::InvalidRequest but nothing was raised /home/wcolaiuta/work/thin/thin.git/spec/request/mongrel_spec.rb:11:May be something wrong with my local setup, so I am going to try and reproduce on another machine.
-
macournoyer
hey Wincent if you would like to help w/ this I can give you a github invite so you can fork the Thin repo. It's much better then patch ping-pong.
Just send me your email.
-
Wincent Colaiuta
Sure, it's win at wincent dot com.
-
Wincent Colaiuta
Ok, on the subject of the failing specs, I get some of the failures when I try on another machine. Will open a separate ticket seeing as it's really a separate issue.
-
Wincent Colaiuta
Ok, opened up ticket #51 0.7.0 (and master) spec failures for tracking down the spec failures.
-
macournoyer
- Milestone changed from 0.8.0 to 0.7.1
I moved all perf specs under spec/perf and exclude them from rake spec. Run through rake spec:perf now.
thx for the suggestion Wincent!
Please Sign in or create a free account to add a new ticket.
With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.