Ed Morley [:emorley] Reporter
	Description • 12 years ago

eg:
https://tbpl.mozilla.org/php/getParsedLog.php?id=16740915&tree=Mozilla-Inbound

{
Generating results file: tp5n_shutdown_paint: 
		Stopped Sun, 04 Nov 2012 10:15:36
...
FAIL: Graph server unreachable (5 attempts)
RETURN:send failed, graph server says:
RETURN:No machine_name called 'talos-r3-xp-096' can be found
RETURN:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 271, in handleRequest
RETURN:    metadata = MetaDataFromTalos(databaseCursor, databaseModule, inputStream)
RETURN:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 63, in __init__
RETURN:    self.doDatabaseThings(databaseCursor)
RETURN:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 92, in doDatabaseThings
RETURN:    raise DatabaseException("No machine_name called '%s' can be found" % self.machine_name)
RETURN:
RETURN:
Traceback (most recent call last):
  File "run_tests.py", line 303, in ?
    main()
  File "run_tests.py", line 300, in main
    run_tests(parser)
  File "run_tests.py", line 276, in run_tests
    talos_results.output(results_urls, **results_options)
  File "C:\talos-slave\talos-data\talos\results.py", line 89, in output
    raise e
utils.talosError: 'Graph server unreachable (5 attempts)\nsend failed, graph server says:\nNo machine_name called \'talos-r3-xp-096\' can be found\n  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 271, in handleRequest\n    metadata = MetaDataFromTalos(databaseCursor, databaseModule, inputStream)\n  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 63, in __init__\n    self.doDatabaseThings(databaseCursor)\n  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 92, in doDatabaseThings\n    raise DatabaseException("No machine_name called \'%s\' can be found" % self.machine_name)\n\n'
program finished with exit code 1
elapsedTime=1831.000000
talosError: Graph server unreachable (5 attempts)
}

TBPL matches on "^talosError:", so only matches the final line.

This meant the failure was mis-starred as bug 547281 for a while before it was noticed that it was just due to an unknown machine name.

I guess we could adjust TBPL to not use "^", but then we'd match multiple times + it would be nice to handle the linebreaks properly too.

	Ed Morley [:emorley] Reporter
	Comment 1 • 12 years ago

https://hg.mozilla.org/build/talos/file/ec4d0dc9513e/talos/output.py#l313

	Ed Morley [:emorley] Reporter
	Comment 2 • 12 years ago

And again:

https://tbpl.mozilla.org/php/getParsedLog.php?id=17104858&tree=Mozilla-Inbound

{
Generating results file: tp5n_main_shutdown_netio_paint: 
		Stopped Fri, 16 Nov 2012 08:00:03
Exception in writing file '['START\nVALUES\ntalos-r3-w7-099,tp5n_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\n0,1804.00,thesartorialist.blogspot.com\n1,1042.00,cakewrecks.blogspot.com\n2,2450.00,baidu.com\n3,1063.00,en.wikipedia.org\n4,524.00,twitter.com\n5,550.00,msn.com\n6,404.00,yahoo.co.jp\n7,1799.00,amazon.com\n8,332.00,linkedin.com\n9,221.00,bing.com\n10,1146.00,icanhascheezburger.com\n11,636.00,yandex.ru\n12,767.00,cgi.ebay.com\n13,1207.00,163.com\n14,722.00,mail.ru\n15,727.00,bbc.co.uk\n16,704.00,store.apple.com\n17,623.00,imdb.com\n18,4271.00,mozilla.com\n19,706.00,ask.com\n20,966.00,cnn.com\n21,959.00,sohu.com\n22,438.00,vkontakte.ru\n23,1024.00,youku.com\n24,494.00,myparentswereawesome.tumblr.com\n25,1004.00,ifeng.com\n26,466.00,ameblo.jp\n27,659.00,tudou.com\n28,326.00,chemistry.about.com\n29,586.00,beatonna.livejournal.com\n30,637.00,hao123.com\n31,1053.00,rakuten.co.jp\n32,228.00,alibaba.com\n33,877.00,uol.com.br\n34,1018.00,cnet.com\n35,468.00,ehow.com\n36,394.00,thepiratebay.org\n37,550.00,page.renren.com\n38,531.00,chinaz.com\n39,923.00,globo.com\n40,751.00,spiegel.de\n41,775.00,dailymotion.com\n42,310.00,goo.ne.jp\n43,328.00,alipay.com\n44,946.00,stackoverflow.com\n45,741.00,nicovideo.jp\n46,345.00,ezinearticles.com\n47,584.00,taringa.net\n48,1812.00,tmall.com\n49,1073.00,huffingtonpost.com\n50,498.00,deviantart.com\n51,691.00,media.photobucket.com\n52,447.00,douban.com\n53,818.00,imgur.com\n54,395.00,reddit.com\n55,592.00,digg.com\n56,323.00,filestube.com\n57,1475.00,dailymail.co.uk\n58,218.00,whois.domaintools.com\n59,620.00,indiatimes.com\n60,633.00,rambler.ru\n61,303.00,torrentz.eu\n62,727.00,reuters.com\n63,719.00,foxnews.com\n64,1488.00,xinhuanet.com\n65,918.00,56.com\n66,1696.00,bild.de\n67,673.00,guardian.co.uk\n68,351.00,w3schools.com\n69,1297.00,naver.com\n70,365.00,blogfa.com\n71,667.00,terra.com.br\n72,368.00,ucoz.ru\n73,760.00,yelp.com\n74,1007.00,wsj.com\n75,568.00,noimpactman.typepad.com\n76,913.00,myspace.com\n77,205.00,google.com\n78,240.00,orange.fr\n79,286.00,php.net\n80,1115.00,zol.com.cn\n81,1038.00,mashable.com\n82,366.00,etsy.com\n83,529.00,gmx.net\n84,575.00,csdn.net\n85,933.00,xunlei.com\n86,566.00,hatena.ne.jp\n87,486.00,icious.com\n88,922.00,repubblica.it\n89,438.00,web.de\n90,538.00,slideshare.net\n91,589.00,telegraph.co.uk\n92,714.00,seesaa.net\n93,520.00,wp.pl\n94,953.00,aljazeera.net\n95,430.00,w3.org\n96,782.00,homeway.com.cn\n97,377.00,facebook.com\n98,587.00,youtube.com\n99,635.00,people.com.cn\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_shutdown_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_normal_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_nonmain_startup_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_startup_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_nonmain_normal_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_normal_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_shutdown_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND']' from results_url: ['START\nVALUES\ntalos-r3-w7-099,tp5n_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\n0,1804.00,thesartorialist.blogspot.com\n1,1042.00,cakewrecks.blogspot.com\n2,2450.00,baidu.com\n3,1063.00,en.wikipedia.org\n4,524.00,twitter.com\n5,550.00,msn.com\n6,404.00,yahoo.co.jp\n7,1799.00,amazon.com\n8,332.00,linkedin.com\n9,221.00,bing.com\n10,1146.00,icanhascheezburger.com\n11,636.00,yandex.ru\n12,767.00,cgi.ebay.com\n13,1207.00,163.com\n14,722.00,mail.ru\n15,727.00,bbc.co.uk\n16,704.00,store.apple.com\n17,623.00,imdb.com\n18,4271.00,mozilla.com\n19,706.00,ask.com\n20,966.00,cnn.com\n21,959.00,sohu.com\n22,438.00,vkontakte.ru\n23,1024.00,youku.com\n24,494.00,myparentswereawesome.tumblr.com\n25,1004.00,ifeng.com\n26,466.00,ameblo.jp\n27,659.00,tudou.com\n28,326.00,chemistry.about.com\n29,586.00,beatonna.livejournal.com\n30,637.00,hao123.com\n31,1053.00,rakuten.co.jp\n32,228.00,alibaba.com\n33,877.00,uol.com.br\n34,1018.00,cnet.com\n35,468.00,ehow.com\n36,394.00,thepiratebay.org\n37,550.00,page.renren.com\n38,531.00,chinaz.com\n39,923.00,globo.com\n40,751.00,spiegel.de\n41,775.00,dailymotion.com\n42,310.00,goo.ne.jp\n43,328.00,alipay.com\n44,946.00,stackoverflow.com\n45,741.00,nicovideo.jp\n46,345.00,ezinearticles.com\n47,584.00,taringa.net\n48,1812.00,tmall.com\n49,1073.00,huffingtonpost.com\n50,498.00,deviantart.com\n51,691.00,media.photobucket.com\n52,447.00,douban.com\n53,818.00,imgur.com\n54,395.00,reddit.com\n55,592.00,digg.com\n56,323.00,filestube.com\n57,1475.00,dailymail.co.uk\n58,218.00,whois.domaintools.com\n59,620.00,indiatimes.com\n60,633.00,rambler.ru\n61,303.00,torrentz.eu\n62,727.00,reuters.com\n63,719.00,foxnews.com\n64,1488.00,xinhuanet.com\n65,918.00,56.com\n66,1696.00,bild.de\n67,673.00,guardian.co.uk\n68,351.00,w3schools.com\n69,1297.00,naver.com\n70,365.00,blogfa.com\n71,667.00,terra.com.br\n72,368.00,ucoz.ru\n73,760.00,yelp.com\n74,1007.00,wsj.com\n75,568.00,noimpactman.typepad.com\n76,913.00,myspace.com\n77,205.00,google.com\n78,240.00,orange.fr\n79,286.00,php.net\n80,1115.00,zol.com.cn\n81,1038.00,mashable.com\n82,366.00,etsy.com\n83,529.00,gmx.net\n84,575.00,csdn.net\n85,933.00,xunlei.com\n86,566.00,hatena.ne.jp\n87,486.00,icious.com\n88,922.00,repubblica.it\n89,438.00,web.de\n90,538.00,slideshare.net\n91,589.00,telegraph.co.uk\n92,714.00,seesaa.net\n93,520.00,wp.pl\n94,953.00,aljazeera.net\n95,430.00,w3.org\n96,782.00,homeway.com.cn\n97,377.00,facebook.com\n98,587.00,youtube.com\n99,635.00,people.com.cn\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_shutdown_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_normal_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_nonmain_startup_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_startup_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_nonmain_normal_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_normal_fileio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND', 'START\nVALUES\ntalos-r3-w7-099,tp5n_main_shutdown_netio_paint,Mozilla-Inbound-Non-PGO,e123e0173185,20121116064858,1353081294\nEND']

FAIL: Graph server unreachable (5 attempts)
RETURN:send failed, graph server says:
RETURN:to determine average from 'test_run_values' for  20772929 - local variable 'values' referenced before assignment
RETURN:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 273, in handleRequest
RETURN:    average = valuesReader(databaseCursor, databaseModule, inputStream, metadata)
RETURN:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 219, in valuesReader
RETURN:    raise DatabaseException("to determine average from 'test_run_values' for  %s - %s" % (metadata.test_run_id, str(x)))
RETURN:
RETURN:
Traceback (most recent call last):
  File "run_tests.py", line 311, in ?
    main()
  File "run_tests.py", line 308, in main
    run_tests(parser)
  File "run_tests.py", line 284, in run_tests
    talos_results.output(results_urls, **results_options)
  File "C:\talos-slave\talos-data\talos\results.py", line 89, in output
    raise e
utils.talosError: 'Graph server unreachable (5 attempts)\nsend failed, graph server says:\nto determine average from \'test_run_values\' for  20772929 - local variable \'values\' referenced before assignment\n  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 273, in handleRequest\n    average = valuesReader(databaseCursor, databaseModule, inputStream, metadata)\n  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 219, in valuesReader\n    raise DatabaseException("to determine average from \'test_run_values\' for  %s - %s" % (metadata.test_run_id, str(x)))\n\n'
program finished with exit code 1
elapsedTime=312.336000
talosError: Graph server unreachable (5 attempts)

TinderboxPrint:send failed, graph server says:

TinderboxPrint:to determine average from 'test_run_values' for  20772929 - local variable 'values' referenced before assignment

TinderboxPrint:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 273, in handleRequest

TinderboxPrint:    average = valuesReader(databaseCursor, databaseModule, inputStream, metadata)

TinderboxPrint:  File "/var/www/html/graphs/server/pyfomatic/collect.py", line 219, in valuesReader

TinderboxPrint:    raise DatabaseException("to determine average from 'test_run_values' for  %s - %s" % (metadata.test_run_id, str(x)))

TinderboxPrint:

TinderboxPrint:
========= Finished 'python run_tests.py ...' failed (results: 2, elapsed: 5 mins, 13 secs) (at 2012-11-16 08:00:04.446567) =========
}

	Ed Morley [:emorley] Reporter
	Comment 3 • 12 years ago

oh ffs more pasted than I realise

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 4 • 12 years ago

can we close this bug?  I believe this is a duplicate of bug 812729

	Ed Morley [:emorley] Reporter
	Comment 5 • 12 years ago

This is about how the part of the message that shows in TBPL isn't the real cause of the failure - and so we should make sure the server response shows up as well.

Clarifying summary.

	Joel Maher ( :jmaher ) (UTC -8)
	Comment 6 • 8 years ago

we don't upload to graph server anymore (except for mozilla-beta)