tenshi_kanade's blog

SPOJ — What the...?
By tenshi_kanade, history, 8 years ago, In English

I want to tell a curious story that happened to me while solving a problem on SPOJ. It happened a few days ago.

  • Solved a problem. Submitted the code. Result: WA.
  • I, being pretty sure that my solution was correct and knowing that SPOJ's checker is an abomination, decided to resubmit the exact same code, only seconds apart from my first submission. Result: AC.
  • Not happy enough, I decided to test the unstable system of SPOJ. So I submitted the same code, for the third time. As I expected, the junk that SPOJ use went haywire again, and result was WA.

This was obviously not the first time I had problems with SPOJ and its rusty short-circuited servers. Every single time I want to solve a problem there, I know what's coming. I expect lots of unstable testing and false WA's.

So, I've made up my mind. From now on, or at least until SPOJ staff decide to do something about that junkyard they use, I'll only use SPOJ if I want to solve a problem that's not on any other online judge. Nothing will give more headaches, not even UVA with its ridiculous parsers.

OK, I just wanted to share my thoughts. Let's hope I don't get so many downvotes for my shitpost. Have a nice day!

  • Vote: I like it
  • +33
  • Vote: I do not like it

Comments Comments (17)
| Write comment?
»
8 years ago, # |
  Vote: I like it +10 Vote: I do not like it

May be the tests randomly generated? Maybe you got lucky once or the code breaks in a very specific test. Or maybe, your code uses randomness somewhere and you got lucky. Anyways, SPOJ is crappy and should be rewritten. Since I started programming the page hasn't changed much (only visually, and we almost don't care about that..) and surely needs refactoring.

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Tests generated randomly would be a disaster. No online judge should do that. And the problem was a simple segment tree problem, so no randomness in the code either. It's just, as you said, SPOJ's crappiness.

    I really like the new interface on SPOJ, but unluckily, the system behind it is still the same horror. Let's hope they decide to update it soon, because the page has a really wonderful problemset.

    → Reply
»
8 years ago, # |
  Vote: I like it +4 Vote: I do not like it

I am curious..how would you find a similar problem as that on SPOJ, on any other online judge?

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    That depends on the problem. A lot of problems, for example some from past IOI's or ICPC Finals, are on many online judges. Some others, on the other hand, are available only on SPOJ. If I come across some nice problem from the second lot that I really want to solve, then I'll have to bear with SPOJ, have a couple of pills at hand and try to suffer as little as possible.

    → Reply
»
8 years ago, # |
  Vote: I like it +3 Vote: I do not like it

I've been using SPOJ and UVA for practice, but they obviously have their shortcomings, as you've mentioned.

Do you have any other OJ recommendations?

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    Codeforces :)

    → Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Code Chef, Hacker Earth, Hacker Rank. They have nice variety of problems, a nice interface, good language support and reliable testing system.

    → Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    You can try LightOJ. There are only around 430 problems. But all the problems have nice format and except for some really easy problems, others are interesting.

    → Reply
»
8 years ago, # |
  Vote: I like it +37 Vote: I do not like it

Are you sure that you are receiving wrong WAs and not wrong ACs? I don't know how it works but maybe you have some undefined behaviour which may result in random things happening?

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +13 Vote: I do not like it

    This would be my first guess too.

    → Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    I'm pretty sure my solution was correct. I tested it against over 20 test cases available at SPOJ Toolkit and it produced correct answers for all of them. Besides, this is not the first time I've received false WA's on SPOJ. The other day, for instance, a friend of mine submitted his code using different version of C++, and got Accepted with only one of them. His code was correct.

    → Reply
    • »
      »
      »
      8 years ago, # ^ |
        Vote: I like it +23 Vote: I do not like it

      I've never seen false WAs happen on SPOJ so far. I'm much more inclined to believe the undefined behavior theory. Care to post the code you're talking about?

      → Reply
      • »
        »
        »
        »
        8 years ago, # ^ |
          Vote: I like it +5 Vote: I do not like it

        Yes, of course. This is the code for the problem Halt The War -> C++ Code.

        I tested it a few moments ago and now it gives Accepted everytime, but somehow, the other day it gave 2 WA's.

        → Reply
        • »
          »
          »
          »
          »
          8 years ago, # ^ |
            Vote: I like it +8 Vote: I do not like it

          I think the problem could be in the segment tree size, in general it should be , but in your code, when you push from a leaf, the new node can be out of bound, so it's safer to set the size in .

          Not sure if this can lead to a WA.

          → Reply
»
8 years ago, # |
  Vote: I like it +34 Vote: I do not like it

As I have a lot of experience in SPOJ (~4 years), I ever have similar problem like this one on this problem (http://www.spoj.com/problems/SMALLM/), you can scroll down to see my comments there and problem setter respond it is really curious thing and reported to SPOJ admins, after that the long discussion continue on email and finally the problem turn out a GCC bug, not SPOJ.

screenshoot

Speaking of server unstability, yes I remember some of SPOJ judging machine has speed unstability so if I submit code multiple times it will show different running time depending on which machine my code run, so if I resubmit my slow AC code sometimes it will have TLE, now after SPOJ migrated to new cube cluster, the effect is significantly reduced as the machine is more stable. I also remember there is a discussion on SPOJ forum about one of the machine has a compiler/interpreter that is not installed properly (now fixed). I don't know if it's possible to happen with WA verdict. Maybe like P_Nyagolov said your code has some undefined behavior(?) If you're sure that your code is correct but getting WA, you can tell problem setter by leaving comment on that problem (so maybe problem setter can fix it or EBs can warn problem setter and edit/remove that problem) or report it directly to [email protected] (will be directly responded by SPOJ admin)

SPOJ has 500+ problem setters and have 5000+ public problems (source: http://www.spoj.com/ranks/psetters/) yet it has only 6 EBs and some busy admins :p so it's very likely to have some SPOJ problem untested and problem may occur, it's not easy to maintain such large site with small team, hope you understand and your help (by just reporting problem) will be appreciated.

By the way I'm one of 6 SPOJ EBs (Editorial Board) if you have trouble with SPOJ public problem (wrong constraint, wrong checker/judge, wrong test data, ...) you can try to report it to me via email (otherwise the respond will be slow or ignored), please explain with info as detailed as possible (such as problem ID, submission ID, what's wrong, what's the right thing should be, suggestion, ...) and please report it only if you're sure there is something wrong with that problem. I'll try to fix that problem and warn problem setter, and if I can't or too busy I'll redirect your message to other SPOJ EB and maybe SPOJ admin.

Have a nice day!
Tjandra

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    Thank you very much for your support. It happened many times in the last few days that the same code gave TLE for one submission and AC for another one just moments later/before. Maybe that's because, like you said, the code is run on different judging machines each time.

    But I believe the main problem with SPOJ's checker, just like UVA's, is that it doesn't allow for different formatting (i.e. space instead of new line, extra space, etc.). A lot of times, I had to add an extra line at the end of the output, or remove it, in order to get AC. That's just unacceptable. That should really be looked into.

    → Reply
    • »
      »
      »
      8 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Default judge of SPOJ is "ignore extra whitespace". It is the fault of problem setter if you have to add newl to get AC. For me I never do problems if I dont recognize PS account, to avoid these.

      → Reply