
did: 4 4, using patterns: 2x3, 3x2, 3x3, 3x4, 4x3
this loop took 5805.0047
allst took 5979.7224

1. here is the plan now: feb 15th
mxn generator is working so well.
2. i need to apply it iteratively
3. once i have a whole bunch of bad shape data (only keeping the reasonable ones; i.e. of size 4x4 don't store ones which embed 3x3 illegals)
then i can apply that to the board, and roomsolving, touching the relevant areas.
for the board it can identify more closely the bad startsqs etc.
for rooms it should speed up solving by a ton.
how far will i ever go?
from observation it would at least cut off a lot in rooms.

hmm, now what about a better version of this: in addition to saying yea/nea on any shape,
for the yea ones it also returns the forced hints.  then at the beginning of the game i overlay those
that would let me solve this type of shape once and for all.

X..X
X..X
X..X - it would remove the intermediate left/right hints which can never happen
plus it would remove a lot of other hints.

1. hypotheitcal solver - for a given room
2. go until deadend is so bad - when it cuts off a room, it should just check that one right away rather htan doing a long other path
3.

0. where am i leaving off, evening of the 8th?
1. one of my recentchanges broke room solving very badly.

todo feb 6:
1. check out setting default hintroom reqstatus to 1, not 2.
2. check bug on 18
3. check deleting old rel/abs/mrgrooms once they're dated.
4. and the big idea - replace reqrooms with their hintroom versions if those are better.
--i.e. grow into forced paths
5 "use hash" is disabld atm because it was randomly turning on on tailslevels

bug: tails128 75 bad gate in a room on the left.

0. level up my tooling.

1. what is the biggest problem? visualization
2. then i can get back to where i was before.

today todo:
1. image data format dumper and pull that stuff out
2. would be neat to have a nice with-arrows visualizer
3. would be neat to have an opposite path visualizer
4. what was my idea last night about checking - i should examine the room solver and see if i can speed it up 10x.  why is it slow still?

1. disabled relhint room adding cause they werea all getting removed for some reason.  if it's below an absroom it shouldn't die but it does and it's 1am.'

instrumentation goals:

tonight:
#1. get reasonable logging again, by level, with general stuff.
#2. rewrite print to go to another log, by level.  
#-also dump better stuff there per room.  amount alocated, etc.
##plus big time savings in here.
3. 

#1. note how successful my timing allocation for rooms is
2. be able to restart after getting cache information, before dynamic run, to clear out ram.
3. measure & save ram usage again, in a better way (split up memlog)
4. test whether in the following solution path, solving e is faster than directly solving it: a,b,c,d,e
5. time static mode & dynamic mode
6. pipe my output to a file, not to the console, so i can track better!.
7. digraph visualizer & solver!?
8. save image data to file, not to slow bmp, and process later.

board.smh(otherrm);board.smh(rm);board.smh()

2 new good ideas:
1. reqtunnel-hints - check them out
2. roomsolver - if you isolate an odd number of reqgates during roomsolving, it's unsolvable
3. if an area around an optgate gets solved, how about just rmeoving the optgate?
4. when making combined hintrooms, i shouldn't combine across hint-barriers.'

check later:get_room_starts_from_hints
flow_to_req_gates:190 isreqtunnel speed., function 31

0. hmm, a common dumb type of work:

1. remove room sols
-trigger hint fixing followed by redoing something
--hint fixing triggers redoing room sols
---that's all resolved
-fall back to reslve the original hint ( but based on the room hintnum which is dumb)

There should be some kind of crosscutting dict of work done keyed by hintcount. and if you are booking work which has already been done at a lower hintcount, just skip that work.

1. don't regenerate rooms which are 100% hintsolved? no it still matters.
2. delete hintrooms at the end of every round?  are they useful?
3. it'd be nice to know which things helpd
4. the progressively increasing check-touched thing adam has is pretty awesome.

1. holy fucking shit.  static pregeneration of "illegal regions" similar to meta-alleys is fucking awesome.  useful everywhere.
i can just precalculate a shit ton of them and quickly compare them to the board to generate extra illegals.  it's ideal. it'll be like i'm as fast as adam's code to do shit, but all frontloaded.

3. bug on 1073, and also tails64x7.  investigating
2. solved bunch of logic errors.


logic error on the right side of 1066 - why didn't it cancel the remainder of the starts in the right group once the end was set up there? wtf.

1065 was taking like this:

and for the first few was taking quite a while too.



(py3-64)[bigchange][/cygdrive/f/proj/coil2] python split.py
18.76562M +18.8M starting split.py hints=0;
18.99219M +0.2M starting main hints=0;
one_sol = 1
static_timelimit = 10
check_touched_timelimit = 0.2
use_hash = 1
levelnum: None
18.92578M -0.1M starting do in order hints=0;
18.95312M +0.0M starting split.py hints=0;
19.01562M +0.1M starting do level variant None, tails:False, tailsnum:0 hints=0;
22.98438M +4.0M before setup board hints=0;
23.05078M +0.1M board - about to reset hints=0;
23.06250M +0.0M board - copied rows. hints=0;
50.19922M +27.1M board - set orig open hints=0;
53.00781M +2.8M board - set orig open sorted hints=0;
82.16406M +29.2M board - added reqtunnel, deeptunnel stuff. hints=0;
82.17188M +0.0M board - reset hints=0;
82.17188M       board - about to load hints. hints=0;
82.17578M +0.0M board - no file to load hints from. hints=0;
82.17969M +0.0M board - loaded board hints. hints=0;
82.17969M       board - 0/231947 setup board original hints hints=0;
95.31250M +13.1M board - 10000/231947 setup board original hints hints=0;
109.82031M +14.5M board - 20000/231947 setup board original hints hints=0;
125.17188M +15.4M board - 30000/231947 setup board original hints hints=0;
139.03125M +13.9M board - 40000/231947 setup board original hints hints=0;
155.85938M +16.8M board - 50000/231947 setup board original hints hints=0;
169.66797M +13.8M board - 60000/231947 setup board original hints hints=0;
183.71484M +14.0M board - 70000/231947 setup board original hints hints=0;
197.55859M +13.8M board - 80000/231947 setup board original hints hints=0;
217.37891M +19.8M board - 90000/231947 setup board original hints hints=0;
231.21094M +13.8M board - 100000/231947 setup board original hints hints=0;
245.03125M +13.8M board - 110000/231947 setup board original hints hints=0;
258.85938M +13.8M board - 120000/231947 setup board original hints hints=0;
272.96484M +14.1M board - 130000/231947 setup board original hints hints=0;
286.84375M +13.9M board - 140000/231947 setup board original hints hints=0;
300.70312M +13.9M board - 150000/231947 setup board original hints hints=0;
314.55469M +13.9M board - 160000/231947 setup board original hints hints=0;
328.37891M +13.8M board - 170000/231947 setup board original hints hints=0;
354.24219M +25.9M board - 180000/231947 setup board original hints hints=0;
368.34375M +14.1M board - 190000/231947 setup board original hints hints=0;
382.16016M +13.8M board - 200000/231947 setup board original hints hints=0;
395.96094M +13.8M board - 210000/231947 setup board original hints hints=0;
409.78516M +13.8M board - 220000/231947 setup board original hints hints=0;
423.61719M +13.8M board - 230000/231947 setup board original hints hints=0;
615.10547M +191.5M board - after orig board hints hints=0;
615.10547M       board - about to add starts & ends hints=0;
631.15234M +16.0M board - takens redone. hints=0;
631.15234M       board - init done - starting static. hints=0;
631.15234M       starting static analysis 1065 hints=0;
631.15234M       static - before making reqrooms hints=0;
631.15234M       static - making reqrooms. hints=0;




5-546 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.9917
42582/65573 optrooms-72dc1a 076-162 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0004 sols (c:4 s:0 e:0 b:0) took 0.1239
42583/65573 optrooms-a97ab6 076-173 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0004 sols (c:4 s:0 e:0 b:0) took 0.1721
42584/65573 optrooms-26a0d9 077-394 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.2922
42585/65573 optrooms-49638a 077-430 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.5262
42586/65573 optrooms-fdfb08 077-346 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.2052
42587/65573 optrooms-49a0e3 077-435 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 1.4401
42588/65573 optrooms-5711dc 080-046 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.5506
42589/65573 optrooms-5711dc 080-170 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.3291
42590/65573 optrooms-5711dc 080-311 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.1945
42591/65573 optrooms-5711dc 080-432 007sq 1/2 (dificulty:26.7) 85.7% hs (008 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0005
42592/65573 optrooms-a97ab6 081-562 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.1818
42593/65573 optrooms-027456 083-203 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.3426
42594/65573 optrooms-49a0e3 084-380 007sq 1/2 (dificulty:240.7) 0.0% hs (024 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.2815
42595/65573 optrooms-52afce 085-335 007sq 1/2 (dificulty:282.5) 0.0% hs (026 h) | M:c | C:c | Cant:seob | tl=1.000 0004 sols (c:4 s:0 e:0 b:0) took 0.0939
42596/65573 optrooms-5711dc 085-494 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.6929
42597/65573 optrooms-a97ab6 086-573 007sq 1/2 (dificulty:167.1) 14.3% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.3633



252 static => how much ram? up to 96, down to 70, up to 112. finally 85

1. testing moving allposs to be a property.
--level 252 with new difficulty stuff. rooms made, 34MB
20.22266M       static - making reqrooms. hints=0;
23.62109M +3.4M static - grouped. 90 hints=0;
23.73047M +0.1M static - 90 reqrooms made in 0.738 hints=0;
23.75781M +0.0M static - after reqrooms hints=0;
23.75781M       static - making optrooms. hints=0;
31.69531M +7.9M static - 992 optrooms made in 1.254 hints=0;
31.73438M +0.0M static - after optrooms hints=0;
31.73438M       static - cleaning rooms hints=0;
31.73438M       static - cleaned rooms hints=0;


52.87109M       solved roomnum 0, 85937e, sols: 2 hints=0; solstep caching; 1959 steps saved.
1/50 hintrooms-ceb08e 058-037 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
2/50 hintrooms-d5d1b3 048-037 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
3/50 hintrooms-f524ac 023-046 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0013
4/50 hintrooms-f524ac 063-037 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0011
5/50 hintrooms-f524ac 016-082 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
6/50 hintrooms-d5d1b3 067-027 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
7/50 hintrooms-d8af70 026-041 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
8/50 hintrooms-85937e 068-047 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0015
9/50 hintrooms-d8af70 019-022 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0012
10/50 hintrooms-581685 061-013 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0013
11/50 hintrooms-d5d1b3 063-041 008sq 4/0 (dificulty:51.4) 50.0% hs (012 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0011
12/50 hintrooms-58076b 049-065 012sq 2/0 (dificulty:68.2) 16.7% hs (022 h) | M:c | C:c | Cant:seob | tl=1.000 0001 sols (c:1 s:0 e:0 b:0) took 0.9010
13/50 hintrooms-bdffff 053-003 025sq 4/0 (dificulty:71.4) 100.0% hs (025 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0062
14/50 hintrooms-b9997a 068-054 027sq 4/0 (dificulty:77.1) 100.0% hs (027 h) | M:c | C:c | Cant:seob | tl=1.000 0001 sols (c:1 s:0 e:0 b:0) took 0.0082
15/50 hintrooms-a52f57 022-060 025sq 6/0 (dificulty:120.7) 100.0% hs (025 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0176
16/50 hintrooms-cb27ca 037-074 029sq 6/0 (dificulty:140.0) 100.0% hs (029 h) | M:c | C:c | Cant:seob | tl=1.000 0001 sols (c:1 s:0 e:0 b:0) took 0.0189
17/50 hintrooms-7d7375 023-080 013sq 6/0 (dificulty:148.5) 46.2% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0107
18/50 hintrooms-4fd242 004-011 013sq 6/0 (dificulty:148.5) 46.2% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0076
19/50 hintrooms-7ea6c2 073-027 013sq 6/0 (dificulty:148.5) 46.2% hs (020 h) | M:c | C:c | Cant:seob | tl=1.000 0002 sols (c:2 s:0 e:0 b:0) took 0.0039
20/50 hintrooms-7e79e4 045-040 013sq 6/0 (dificulty:163


79.82422M       static - broke from solve rooms loop hints=0; solstep caching; 2257 steps saved.
SAVED to output/0252\0252-taken-static-starts-00083.png 0.6420
SAVED to output/0252\0252-taken-static-ends-00001.png 0.6260

{(16, 15), (16, 14), (17, 16), (17, 14), (15, 15), (17, 15)}

py2
23.82031M +0.2M dynamic start setup hints=0; solstep caching; 633 steps saved.

py3
24.19531M +0.8M dynamic start setup hints=0; solstep caching; 671 steps saved.

http://deeplearning.net/software/theano/tutorial/python-memory-management.html

0. future plan: no more levels until i can solve things really well in ~300mb ram, so that it'd be reasonable to get to 1k x 1k sizes in a rush.
-testing plan: try smaller tails levels and look at why ram use is staying high even after returning from room solver stuff.  I should be able to create, solve, then delete rooms, and keep doing that
-in an interesting & useful way,  with algos, and have it all be free & with the program able to manage its own memory.
-don't just switch to py64 to delay addressing this problem.
1. today: RAM issues
--idea: what if it's because of mkpos2room linking to the rooms, not to the room hashids?  but if we cna't handle it well then no.

2. later - general room solving examination
3. later: read my old notes and ideas and make a good list of the ones that are still remaining
-bumps is easy, for priority - similar to gate order which i just redid very successfully.
-hintrooms
--do_hintroom_gates

63063/64134 - reqrooms-a40e62 286-375 039sq 14R 0O 100.0% hs (053 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 1.7567
63239/64134 - reqrooms-f5d06c 162-030 051sq 14R 0O 100.0% hs (065 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 2.8109
63260/64134 - reqrooms-6b3a85 395-380 053sq 14R 0O 100.0% hs (067 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 2.4943
63310/64134 - reqrooms-080315 482-342 058sq 14R 0O 90.3% hs (080 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 5.8965
63319/64134 - reqrooms-69536a 207-057 059sq 14R 0O 56.2% hs (125 h) | M:c | C:c | Cant:seob  | tl=3.000 0024 sols (c:24 s:0 e:0 b:0)  took 8.4959
63320/64134 - reqrooms-fdff9c 528-225 059sq 14R 0O 94.5% hs (077 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.4084
63353/64134 - reqrooms-9e70e6 371-148 062sq 14R 0O 100.0% hs (076 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 5.2033
63364/64134 - reqrooms-0ea291 131-213 063sq 14R 0O 100.0% hs (077 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0510
63374/64134 - reqrooms-14cd92 183-098 044sq 16R 0O 66.7% hs (082 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.1700
63375/64134 - reqrooms-efb194 065-475 064sq 14R 0O 89.7% hs (086 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0377
63388/64134 - reqrooms-21426c 402-221 065sq 14R 0O 100.0% hs (079 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 5.4684
63410/64134 - reqrooms-428b62 010-400 068sq 14R 0O 95.1% hs (086 h) | M:c | C:c | Cant:seob  | tl=3.000 long perm res: 180 long perm res: 180 long perm res: 180 long perm res: 180 5040 sols (c:5040 s:0 e:0 b:0)  took 4.9859
63411/64134 - reqrooms-50d8f0 084-272 068sq 14R 0O 78.0% hs (104 h) | M:c | C:c | Cant:seob  | tl=3.000 5880 sols (c:5880 s:0 e:0 b:0)  took 4.9136
63421/64134 - reqrooms-27325f 280-314 069sq 14R 0O 91.6% hs (091 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 5.0745
63422/64134 - reqrooms-579642 268-196 069sq 14R 0O 95.2% hs (087 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 4.8380
63445/64134 - reqrooms-2d29cb 513-103 071sq 14R 0O 85.9% hs (099 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0353
63456/64134 - optrooms-05c6f5 335-419 033sq 3R 5O 2.4% hs (358 h) | M: | C:cso | Cant:eb  | tl=3.000 =>0.001 (impl)   took 0.0026
63459/64134 - reqrooms-9ec796 308-288 073sq 14R 0O 100.0% hs (087 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0195
63460/64134 - reqrooms-ebdce8 460-173 073sq 14R 0O 100.0% hs (087 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 4.0129
63465/64134 - reqrooms-16fa2a 067-353 074sq 14R 0O 88.6% hs (098 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.1305
63466/64134 - reqrooms-fdb45e 493-123 074sq 14R 0O 100.0% hs (088 h) | M:c | C:c | Cant:seob  | tl=3.000 2520 sols (c:2520 s:0 e:0 b:0)  took 4.1612
63495/64134 - reqrooms-18ddb9 501-101 079sq 14R 0O 59.1% hs (159 h) | M:c | C:c | Cant:seob  | tl=3.000 6720 sols (c:6720 s:0 e:0 b:0)  took 7.4583
63510/64134 - reqrooms-0be6df 277-151 081sq 14R 0O 92.6% hs (102 h) | M:c | C:c | Cant:seob  | tl=3.000 1680 sols (c:1680 s:0 e:0 b:0)  took 2.1672
63524/64134 - reqrooms-fc4b9c 379-017 083sq 14R 0O 92.8% hs (104 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.1299
63537/64134 - optrooms-cc79e1 342-128 045sq 9R 3O 21.1% hs (286 h) | M:c | C:c | Cant:seob  | tl=3.000 0006 sols (c:6 s:0 e:0 b:0)  took 8.0411
63538/64134 - reqrooms-99ab48 543-103 085sq 14R 0O 86.9% hs (112 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.1798
63543/64134 - reqrooms-d9f9f7 534-119 086sq 14R 0O 96.0% hs (104 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 5.7185
63555/64134 - optrooms-d055af 512-052 028sq 5R 5O 100.0% hs (038 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0 b:0)  took 4.3952
63557/64134 - reqrooms-4eba55 339-175 088sq 14R 0O 35.3% hs (186 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0422
63558/64134 - reqrooms-ede774 309-384 088sq 14R 0O 80.4% hs (124 h) | M:c | C:c | Cant:seob  | tl=3.000 5376 sols (c:5376 s:0 e:0 b:0)  took 5.0305
63560/64134 - reqrooms-63287f 066-361 089sq 14R 0O 82.5% hs (129 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.2218
63567/64134 - reqrooms-f8d052 391-228 070sq 16R 0O 100.0% hs (086 h) | M:c | C:c | Cant:seob  | tl=3.000 13440 sols (c:13440 s:0 e:0 b:0)  took 10.4563
63568/64134 - reqrooms-71f710 270-105 090sq 14R 0O 96.2% hs (108 h) | M:c | C:c | Cant:seob  | tl=3.000   took 3.0433
63571/64134 - reqrooms-1242b4 030-063 091sq 14R 0O 96.2% hs (109 h) | M:c | C:c | Cant:seob  | tl=3.000 2520 sols (c:2520 s:0 e:0 b:0)  took 3.6127
63572/64134 - reqrooms-813a1e 106-398 091sq 14R 0O 100.0% hs (105 h) | M:c | C:c | Cant:seob  | tl=3.000 5040 sols (c:5040 s:0 e:0

Did today:
1. added multimove_hint which is awesome
2. added efficient forced paths
3. fixed up hint & sol loading
4. things are way faster.  can solve 128x most of the time super fast
5. there's still a problem with clean boards.  sometimes i can't solve them at all.
6. also fixed up a serious bug in alley detection
7. also fixed a serious bug in check touched detection - waiting to find out how well it works on 1064.

depth:10921 dt:128501 ********************* 4194303

depth:10922 dt:128501 ********************* 4194303

depth:10922 dt:128498 ********************* 4194303
Traceback (most recent call last):
  File "F:\proj\coil2\split.py", line 263, in fork
    if forkres==0:
  File "F:\proj\coil2\split.py", line 263, in fork
    if forkres==0:

keep going into super multimove (obeying hints)

depth:05595 dt:175097 ***********
zsh: segmentation fault (core dumped)  python split.py

0. hmm, for clean-solvable reqrooms, i admit i have to allow both solutions in them.  but do i have to allow internal, non-exit-order-changing both solutions?  level 127 sucks.

1. invesigate 100% hintsolved rooms which still don't get solved. weird.
2. loading hints from a file
3.

level 50:

11175838 function calls (10920743 primitive calls) in 51.533 seconds
INFO:root:30.539M just deleted allrooms 17. hints=751; sol caching; 304 sols saved.

10389751 function calls (10144134 primitive calls) in 49.098 seconds
INFO:root:30.387M just deleted allrooms 16. hints=751; solstep caching; 237 steps saved.

level 150:

 80575217 function calls (78821480 primitive calls) in 286.672 seconds
 INFO:root:41.410M just removed allrooms from pos2rooms 18. hints=20061; solstep caching; 4941 steps saved.
INFO:root:38.023M just deleted allrooms 18. hints=20061; solstep caching; 4941 steps saved.

72169451 function calls (70572232 primitive calls) in 270.528 secon
INFO:root:48.066M just removed allrooms from pos2rooms 16. hints=20172; sol caching; 14412 sols saved.
INFO:root:48.066M just deleted allrooms 16. hints=20172; sol caching; 14412 sols saved.



level 310:
88355067 function calls (84958086 primitive calls) in 334.943 seconds
INFO:root:81.680M just removed allrooms from pos2rooms 19. hints=10767; sol caching; 10396 sols saved.
INFO:root:75.043M just deleted allrooms 19. hints=10767; sol caching; 10396 sols saved.


90177958 function calls (86549081 primitive calls) in 355.940 seconds
INFO:root:73.879M just removed allrooms from pos2rooms 19. hints=10400; solstep caching; 1573 steps saved.
INFO:root:65.922M just deleted allrooms 19. hints=10400; solstep caching; 1573 steps saved.

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   226850   31.071    0.000   31.144    0.000 sol_management.py:63(getsols)
       26   23.457    0.902   23.457    0.902 {nt.system}
    15006   23.279    0.002   30.780    0.002 util.py:83(next)
99748/10099   22.011    0.000  145.413    0.014 room_solver.py:600(make_subrooms_from_current_state)
255161/141926   14.605    0.000   51.520    0.000 room_solver.py:455(go_until_deadend_with_path)
      120   12.410    0.103   12.410    0.103 {method 'encode' of 'ImagingEncoder' objects}
62806/3566   11.740    0.000  157.261    0.044 room_solver.py:154(all_breakdown)
3584630/557175   10.511    0.000   10.511    0.000 split_utils.py:339(off

import ctypes

class MEMORYSTATUSEX(ctypes.Structure):
    _fields_ = [
        ("dwLength", ctypes.c_ulong),
        ("dwMemoryLoad", ctypes.c_ulong),
        ("ullTotalPhys", ctypes.c_ulonglong),
        ("ullAvailPhys", ctypes.c_ulonglong),
        ("ullTotalPageFile", ctypes.c_ulonglong),
        ("ullAvailPageFile", ctypes.c_ulonglong),
        ("ullTotalVirtual", ctypes.c_ulonglong),
        ("ullAvailVirtual", ctypes.c_ulonglong),
        ("sullAvailExtendedVirtual", ctypes.c_ulonglong),
    ]

    def __init__(self):
        # have to initialize this to the size of MEMORYSTATUSEX
        self.dwLength = ctypes.sizeof(self)
        super(MEMORYSTATUSEX, self).__init__()

stat = MEMORYSTATUSEX()
ctypes.windll.kernel32.GlobalMemoryStatusEx(ctypes.byref(stat))

print("MemoryLoad: %d%%" % (stat.dwMemoryLoad))

1. load hints from pypy run
2. 

made subrooms
about to fix subroomres


1. in-out ordering must be alternatingg in clean rooms!  why?  this also applies to implied things.  so if there is a subloop (within a room) you can guess the ordering & force the path most of the time.  so weird!
2. if all reqrooms are clean, then they all must be either clean or both.  there can_start and can_end must be false!  test this on a room which no seperators were found on!
3. question: is a.intersection(bb) really slower than not a.isdisjoint(b)

1. if a sq gets hint solved, i also should notify t

he rooms along forced paths of it, because they may need to be better pruned too.  or just end a final better prune step?
2. huh.  instead of creating pararooms, i can just nuke pararoom side-ways hints!
3. it wouldn't actually be that hard to just "try" some starts during static mode & eliminate them.  but would that help me?
4. when solving huge rooms, even if they have like 98% hint solved, if they have a lot of reqgates it can still time out even given 2s.  maybe something about in the ordering of subroomreses which knows about bumps?  but they're independent, it shouldnt happen anyway...
5. when you have two ils but don't know which is which, it might be faster to just try each one as a start rather than try to go as far as i can without declaring.
6. can i do ative culling of starts?
7. what is the RAM problem?  dies around 1.7-1.8.  why?
8. can i shink output images & have it look okay still?
9. to do 16x board size i need to use 100mb ram on level 1064.  far cry from where i am now
10. would bump pruning help?
11. why aren't more rooms being removed even when i have 100% hintsolved?


1. shrink output
2. save hints to a file & load it at start. would save a lot.
3. save ram use by hintcount?
4. timing stuff should display hints/second / some kind of solve rate.
--and i should allocate timelimits in a smarter way.
5. why doesn't ram use go down?  should i try gc? what are the big objects I have?
6. active cleaning?
7. interesting - if a room is opt, with opt+req even, and one opt, then we know that is actually a req.  and i can remove hints related to it.

good::: 1. rooms should not include gates - just poss.  and derive the gates from the inbound hints
2. cache rooms by hints! not by shape.  this would enable caching again.  good!

hmm another divider position:
X..X
X..X
..XX
X..X
X..X
the middle section is interesting.

f/e in 71 lower left, the edges of it have starts.  but is the interior of the room solutions correctly not including starts?
1. I should really add in my "mark solved" business for prioritizing starts.  this is crazy.
2. I should use solutions (& hints?) by hashid
3. figure out why ram use is so high
4. verify that allsols_cache is getting cleared every time.
5. use is_disjoint for faster stuff.
6. if an oddset has a force hint dividing it, then 



idea: I could pregenerate the doing initial placement

2. what if i tested removing certain hints in the static stage, and then if removing them cascaded to a contradiction, i just removed them from staic global options?
3. what about saving the whole path of a sol and then detecting the first hint removal of a forced path?
4. the center square of this configuration is acutally super interesting.

todo which is awesome: if this is guaranteed not odd, then you can remove crossing hints in the middle! very interesting.
XXXXX
XX..X
X...X  you can predefine it as a "must pass through"
X..XX
XXXXX
hmm, i should remove hints like 0,1 from it.  well, either must pass through or odd.  i guess you could start at A and go up and the nsolve the rest.


1. fix bugs
2. fix check_solved
3. fix room making & targetting
4. fix display while running.
5. okay level 5 is buggeed again.  geez.

for room solving:
a: my terminology about odd pos sets is wrong.
odd actually means "unclean" the way i'm using it
and i am currently not storing the real information about "odd" anywhere.

2. backsolving is actually interesting.  i.e. if we know the end just try to fit it in in reverse

21st evening 11pm:
1. testing from tails 0-20 after my speed fixes in taken, etc. related to placement.  hope it works!
#2. room solution tester working theoretically and a bunch of data stored.


if we know "can take both" is false, and the first subroom took something, then we can check for the leftover subroom having anything clean first.
!!0. simple application of room in outs - if odd/even both taken and a room has 2 solid used gates and one optgate, then kill the optgate.  actually you can just remove the hints underneath and it'll be okay.'
1. I should check if rooms which are can_odd==False overlap odd regions.  that is useful information.
2. there is a bug in the new hint making mechanism that shows up on tails 35-110
--it seems to save time, but don't know if the extra hints that it outputs are costing me more later.
0. hint-bordered areas - treat hints as walls.  a given whitespace area with maybe 1 real gate and some hints.

3. there is some duplicate work when setting the timelimit up and down.
pruning: if there is an un-totall-solved region which only has one "in", then that in must come first... duh.
bug: start in a parasqs and clean it all then die at the end dumbly.  should be able to turn out or keep going!

Todo December 12, 2015 day
4. what about this for dynamic work:
--save original hints after static
--for each room save solutions which would happen "without starts" if the start is not in
--at each start, reset hints
--then do a few rounds of cancelling.
--then do the run.  should cut down on run times significantly
--then restore original global hints.
---There'd still be tons of wasted work...
5. life:

