18 replies to this topic

[Dan]

Hey guys,

 

As some of you might remember, back at the beginning of December I released a small app to attempt to gather data to figure out the new algorithm used by the Abandoned Attic.

 

This post is going to explain how the data was collected, and the conclusions that @WaserLave and I came to after hours of analysis.

 

Aim

The aim of this was to determine if there was a reliable way to predict the restock times of the Abandoned Attic, and to see if an algorithm could be designed to determine when this would be.

 

Preconceptions (Things We Already Knew)

• Previously the attic ran on a 7 minute restock timer. This means that if you caught a restock at 00 minutes past the hour, you could rely on the next restock being a multiple of 7 minutes away from that initial restock time. 
• JellyNeo used to have a really helpful page with the previous restock time possibilities here, but it's been replaced by a "we're working on it" sign (hehe  )

Data Collection Method

 

A small test application was deployed to a few user's computers, which was left running for various periods throughout the days this test was run.

 

This app refreshed at a preconfigured interval to check for a restock in the Attic and provided the following features:

• Configured intervals for standard restock checking and post-restock ban restock checking
• Proxy configuration
• Restock ban detection

A successful restock was indicated by there being more items in stock than the previous attic refresh. 

 

If a restock was detected, this was pushed out to a log file, which was then manually sent in to me and then all of this data was added manually to an Excel spreadsheet for analysis.

 

Overall, 806 restocks were detected from a total of 11 users over a 7-day period. There is no way to tell whether we caught every restock.

 

Analysis Of Collected Data & Theorems

 

After we collected data for 50 restocks we began to notice a very similar algorithm to the previous one - at least some noticable commonality between the previous '7 minute interval' and the data we were receiving.

 

A good example of this is in the sample below. 

(NOTE: If you're following this in the spreadsheet, look at row 138 on the Offset Analysis sheet)

 

Here we're going to take a sample of 6 attic restocks, gathered on the afternoon of 03/12/2013:

21:43:58-21:44:18	Napiform
21:44:05-21:44:25	Futurama

22:05:05-22:05:25	Napiform
22:05:06-22:05:26       Futurama

22:26:06-22:26:26	Futurama
22:26:16-22:26:36       Napiform

22:40:20-22:40:40	Napiform

23:15:31-23:15:51	Napiform

23:22:33-23:22:53	Napiform

The pure restock times have been pulled out to the below (with the time between restocks also noted):

Date            Restock Time         Time since last restock
03/12/2013	21:44:05-21:44:18    n/a
03/12/2013	22:05:05-22:05:25    ~21 minutes
03/12/2013	22:26:16-22:26:26    ~21 minutes
03/12/2013	22:40:20-22:40:40    ~14 minutes
03/12/2013	23:15:31-23:15:51    ~35 minutes
03/12/2013	23:22:33-23:22:53    ~7 minutes

We saw this as very significant and decided to watch to see if this pattern continued - as we had also seen the same pattern emerge with some earlier values.

 

To see if we can actually predict the times, we can extrapolate multiples of 7 minutes (ignoring seconds values) from the initial restock we see in this sample (@ 21:44):

Restock Time       Restock seen
21:44:00           [X]
21:51:00           [ ]
21:58:00           [ ]
22:05:00           [X]
22:12:00           [ ]
22:19:00           [ ]
22:26:00           [X]
22:33:00           [ ]
22:40:00           [X]
22:47:00           [ ]
22:54:00           [ ]
23:01:00           [ ]
23:08:00           [ ]
23:15:00           [X]
23:22:00           [X]
...

If restock times were indeed on a 7 minute interval as was the case before, we'd start to see the restocks through the day match up with our list of extrapolated times.

 

Unfortunately (for us, as if this was the case, it would make our lives exponentially easier), this was not the case.

 

The next reliable restock time we saw was at 01:15:xx AM on 04/12/2013.

 

If we try to match that with our extrapolated times, we see the following:

Restock Time      Restock hit
...
23:22:00          [X]
23:29:00          [ ]
23:36:00          [ ]
23:43:00          [ ]
23:50:00          [ ]
23:57:00          [ ]
00:04:00          [ ]
00:11:00          [ ]
00:18:00          [ ]
00:25:00          [ ]
00:32:00          [ ]
00:39:00          [ ]
00:46:00          [ ]
00:53:00          [ ]
01:00:00          [ ]
01:07:00          [ ]
------------------------------
01:14:00          [ ]
------------------------------
01:21:00          [ ]
...

The closest value we have is the 01:14:00 restock time.

 

This means that our 7-minute extrapolation rule was broken, as it had now restocked at 01:15:00 AM, 1 minute after we had expected it to restock.

 

The next restocks were as follows:

Restock Time            Expected Time    Time since previous
01:22:12-01:22:26	01:21:00         n/a
01:57:28-01:57:48	01:56:00         ~35 minutes
02:25:49-02:26:09	02:24:00         ~28 minutes
02:32:50-02:33:10	02:31:00         ~7 minutes
02:46:50-02:47:10	02:45:00         ~14 minutes
03:14:51-03:15:11	03:13:00         ~28 minutes

We can see that the time between potential restocks still appears to be 7 minutes, but there seems to have been an "offset" of +1 minutes added onto our expected value.

 

This pattern repeats itself, and after a certain period of time another minute is added to the offset. This cycles all the way through from 0-6, with an average cycle taking around 18 hours.

 

RECAP

 

So, here's what we know

• Restock times are still based on some sort of a 7-minute timer.
• There are various 'windows' throughout the day indicating what times the attic *may* restock
• These 'windows' do not seem to repeat on a known week-by-week basis. 
• An entire cycle from offset 0 to offset 6 appears to take ~18 hours.
• Before we attempt to tackle the idea of seconds past the minute, we should first get the minute correct.

 

Algorithm (untested, not guaranteed reliability)

 

Based on all of this, I've been able to come up with the following algorithm that should, in theory, work for all intents and purposes.

 

Consider the following pseudo-code:

> Find a restock time by refreshing at a specific interval until we see more items in stock (e.g. 30 seconds)
> Wait until restock time + ~7 minutes, then check for restock
>> If restocked then check items and buy if necessary, restart algorithm from this time
>> If not restocked then check again in ~1 minutes time to see if a restock happens (offset may have incremented)
>>> If a restock is found, check items and buy if necessary, restart algorithm from this time
>>> If not restocked, continue + check again in ~6 minutes time from the last restock time

This should mean that most, if not all, restocks are detected, however, we'd be doing this pretty inefficiently. 

 

Improvements

• If we scaled up to collect more data per restock then we could better tell at what seconds past the minute the attic is going to restock, and analyse any patterns to do with this.
• If this experiment was to be run again I'd definitely write some kind of log parser to get around the manual entry of each restock time brackets into the excel spreadsheet, as this was a real ballache.

We need you!

 

If you found all of this interesting and would like to help contribute, please PM me and I can get you a copy of the raw spreadsheet and we can continue to work together to figure out the new Attic algorithm.

 

Key Contributors

@Dan

@WaserLave

 

Volunteer Data Curators

@Nadrak

@Futurama

@Strategist

@Napiform

@Scot

@Kat

@Nymh

@Nimby

@Adam

 

 

 

Feel free to discuss in this thread.

 

OK Dan, but how do we fix Ambrosia?
 
In order to actually achieve the aim we set out to get, we'd need a couple of things:

• More Data™, and in turn,
• More Users

We essentially would need a guaranteed method to figure out the exact restock times so that we could have a completely reliable dataset to analyse.
 
How do we get that?
 
We essentially have two options:

• Brute force

We could essentially get every restock time by brute force - we could have clients refresh the attic at every second in a day. If we had enough users, we could definitely do this

• Smart

We could attempt to use the algorithm already developed and run multiple clients to attempt to grab the exact, correct restock times. This should, in theory, provide us with the same resultset but with a lower hitcount on neopets. We could utilise some kind of burst of clients for when we think a refresh minute might be.

 

Why did you do all of this?

 

Because I found it fun, and you should too. If you're interested in being part of the war effort, even if you can't math or program, just PM me and I'm sure I can fit you in somewhere..

[best]

Hey Dan, I can help you, please let me know what can I do for you

[Strategist]

Hey Dan, I can help you, please let me know what can I do for you

If you found all of this interesting and would like to help contribute, please PM me and I can get you a copy of the raw spreadsheet and we can continue to work together to figure out the new Attic algorithm.

 

[Eefi]

I'll shoot you a pm.

What I find disturbing is that TNT could just make it entirely random and it's all for naught to try to figure out anything. But as it seems, it's not completely random but that could be bad enough if it was a normal distribution with mean at +7min.

Edited by Eefi, 08 February 2014 - 12:55 AM.

[Dan]

Thanks for the offers of help guys, I'll definitely keep you in the loop when I've figured how we should move forward on this.

 

One more thing:

 

 

One pattern I have sort-of noticed is that the 'offset window' appears to reset every 18 hours, as previously mentioned:

03:46~ - Initial refresh (SEED)
21:43~ - Reset (back to 0)
15:48~ - Reset 
09:18~ - Reset

There's 18 hours between all of those, so in theory we'd have be able to see the same time-periods appear 72 hours after the initial values.

 

This worked the first time round, but unfortunately our data doesn't fully prove this theory as time went on. The reason for this is lack of reliable data.

 

The next we saw were:

05:16~
22:53~
16:44~
10:35~

04:25~
22:59~

These don't match up to the "18 hour" points we saw so clearly earlier, and this is down to two problems:

 

• Lack of data 

Unfortunately it's possible that we missed certain restocks at this time.

• Randomisation of restocks

Even if we gathered ALL of the restocks in the attic, then we still wouldn't be able to see a perfect 18 hour round-trip, and that's because the attic only provides the *POSSIBLE* chance to restock every 7 minutes, it is not guaranteed. This means in order to see exactly how long each offset window is or isn't, we'd need to analyse data for even longer until we saw the restocks before and after the 'offset window' changes.

 

 

Can somebody read this and tell me if I'm making any sense here?   I don't know if I've just turned into a mad scientist.

[Strategist]

Thanks for the offers of help guys, I'll definitely keep you in the loop when I've figured how we should move forward on this.

 

One more thing:

 

 

One pattern I have sort-of noticed is that the 'offset window' appears to reset every 18 hours, as previously mentioned:

03:46~ - Initial refresh (SEED)
21:43~ - Reset (back to 0)
15:48~ - Reset 
09:18~ - Reset

There's 18 hours between all of those, so in theory we'd have be able to see the same time-periods appear 72 hours after the initial values.

 

This worked the first time round, but unfortunately our data doesn't fully prove this theory as time went on. The reason for this is lack of reliable data.

 

The next we saw were:

05:16~
22:53~
16:44~
10:35~

04:25~
22:59~

These don't match up to the "18 hour" points we saw so clearly earlier, and this is down to two problems:

 

• Lack of data 

Unfortunately it's possible that we missed certain restocks at this time.

• Randomisation of restocks

Even if we gathered ALL of the restocks in the attic, then we still wouldn't be able to see a perfect 18 hour round-trip, and that's because the attic only provides the *POSSIBLE* chance to restock every 7 minutes, it is not guaranteed. This means in order to see exactly how long each offset window is or isn't, we'd need to analyse data for even longer until we saw the restocks before and after the 'offset window' changes.

 

 

Can somebody read this and tell me if I'm making any sense here?   I don't know if I've just turned into a mad scientist.

Makes sense to me

[WarezHaxor]

Thing of it is, nothing is truly random. And you're right, you need more data to truly be able to figure it out. You could add a logging function into abrosia connected to a codex database that'll log all restocks all users see...and after a week or two you should have a good sized data set to work with...developing a program to identify the algo would be nice, ill see if I can find some spare source on my old desktop from back in the day for that...makes life a hell of a lot easier when things change again.

Point is, every randomize function derives its result from some formula based on a beginning input number (s)...a lot of games these days use the milliseconds of the current server time for these randomized functions...they're not truly random if you can figure out the input;-)

[Eefi]

Point is, every randomize function derives its result from some formula based on a beginning input number (s)...a lot of games these days use the milliseconds of the current server time for these randomized functions...they're not truly random if you can figure out the input;-)

That's true but the restock time (seconds or milliseconds) is somehow derived from from a PRNG so you have to figure out the algorithm: (milli-)second restock time -> PRNG output -> seed

I doubt the RNG is seeded for every call, it probably gets seeded once its started and they could use some sort of global RNG for all random calls e.g. shop restocks, random events etc. It's easy for them to implement and hard to figure out when you don't even get the plain RNG number.

[WarezHaxor]

Correct, but all algorithms are just that, an algorithm. They are all formulae therefore there's always a way to find them, just makes it easier with a larger data set. If they implemented rs logging into abrosia, and linked it to a db on codex, even if only 100 people restock ib AAA over the course of 2 weeks your dataset would be HUGE with plenty of correlating points to compare to derive the beginning algorithm. The seed isn't always necessary...a function could figure out the time from 2 - 3 restocks plugged into the algo to arrive with where in the process they are at.

[NapisaurusRex]

Can somebody read this and tell me if I'm making any sense here?   I don't know if I've just turned into a mad scientist.

Yes, this makes sense to me.

While I'm not good with numbers and patterns and stuff, I will be volunteering to collect data if we do that again.

[L33T]

I won't be participating in this as I have no non-frozen shells atm, but +1 for this cause.

Edited by L33T, 09 February 2014 - 05:36 PM.

[Bear]

I wish I could donate money for this cause.... I enjoyed our AAAber so much. I don't have the time to help with the data agggghh.

[Dan]

I gather that the intervals are always at the very least 7 minutes and never less?

 

From what we saw back when this was run at the beginning of December, yes.

 

This is what's so ... 'patterny' about it.

[Dan]

 
Some pseudocode to play with:

Refresh every minute/30 seconds on the dot until restock detected (when new items appear or when the in stock of an item goes up)
If restock, then set cycle starting from that minute.

Cycle:
If just begin, pause for 6.5 minutes; else pause for 6 minutes
Refresh using refresh rate for one minute or until restock detected
If restock detected and is in furthest 1/3 of the minute refreshing, shift entire cycle forwardby 0.5 minutes

So what do you think @Dan? This should catch every restock after the initial finding part. Is there anything wrong in my algorithm? 

 

I can't see any initial problems with it, but I don't see what benefit we get using your pseudocode over the one that I posted above. 

[Dan]

 

Good news everyone!

 

Version 2 is on it's way.

 

With the help of @Eefi I've managed to make some serious improvements:

 

• We have a UI! No more editing files to substitute in your username & password. This was the most requested part from the last round of testing, so I figured I'd work hard and get it in for you guys.
• We have a centralised database. Instead of you having to PM me your attic logs, we now have a database sitting on a server of mine that receives data about every restock you see. This also allows us to create some cool stuff like LIVE statistics and LIVE restock times posted onto a web page.
• We have an announcement system. Into the UI I've built in an 'announcement' system that allows me to push out announcements for a new version, any maintenance, along with other bits and pieces.

This is scheduled to release (probably™) Tomorrow.

 

In the mean time, here's some screenshots of the brand spanking new interface.

 

Spoiler

 

 

 

 

(yes, it's windows only, sorry, majority wins!)

[Strategist]

Great progress, @Dan and @Eefi. Thanks for continuing with this. Will be helpful in determining the true pattern for sure.

[Sweeney]

Looks cool, Dan. Nice work

[Dance]

This is awesome! Wish I could participate *glares at macbook* 

[Dan]

This is awesome! Wish I could participate *glares at macbook* 

 
 

Ugh time to pull out the VM again so I can participate
Thanks for your work guys!

Trust you guys to be running *nix!

I might be able to get a command-line version running for Mac/Linux.