14 March 2013

Mysterious cab files fill-up temp folder

Our Labtech RMM recently spit out a warning that a client's 2008R2 server had their disk space filling up fast (Note they are not a premium customer):



Upon closer inspection I found that every hour an unknow process would attempt to write a .cab file of approx 60MB to the Windows temp folder. Checking with Process Explorer I found that it was makecab.exe writing these files. Makecab was invoked by services.exe, so that was a bit of a dead end. I looked through the list of Windows scheduled tasks, but did not find anything that was supposedly run every hour.

I tried to rename the cab files created, adding a .cab extension, but they turned out corrupt. I then used Sysinternals ProcessExplorer to find the source of the cab file, which was tricky, as it would only take a few minutes to actually create the file. The indication was that the source of the cab file (eg the file being compressed) was from the c:\windows\logs\cbs folder.


I learned a bit more about Windows Resource Protection (WRP), which stops programs overwriting essential system files. It keeps its log files in this folder. The SFC.exe program writes the details of each verification operation and of each repair operation to the CBS.log file. The CBS.persist.log is generated when the CBS gets to be around 50Mb in size. CBS.log is copied to cbs.persist.log and a new cbs.log file is started. A bit of Google foo and we determine that the cbs logs would only be useful for serious troubleshooting issues. If the system is running fine, we can delete this file. SFC.exe will create a new one, next time it is run.
So why do we have this mysterious process writing a file here?? It appears that on this server the SFC archive process has not been running for a long tome. On the 9th Feb the system was restarted and the log got recycled. Due to the long period of not being archived the log file grew to 4GB in size. Now the fun begins: Every hour, the archive process tries to create a new .cab file. I now speculate that the file size is larger than  what is supported and the process fails, hence resulting in a partial .cab file that sits in the temp folder, rather than a complete .cab file in the CBS log folder.
I have deleted the offending .cab file and most of the other ones too, just keeping a few recent ones in case we need them. No more mysteries!



22 comments:

  1. Thank you! I ran into exactly the same issue

    ReplyDelete
  2. Thank you very much, a very useful blog outlining the exact root of the issue.
    Space crisis averted :)

    ReplyDelete
  3. Thank you for the good articel. I have basically the same issue. My problem is that the same problem comes up about every two months and there is no way to find out the application that causes this issue... any ideas for a workarround?

    ReplyDelete
    Replies
    1. Just a thought, if your machine is part of a domain with managed Windows Updates, check the WSUS server - you'll probably find that it is happening on that server, too. If it is, it might be that the server is pushing these out.

      Delete
  4. It appears to me that these logs accumulate when you don't reboot after installing windows updates. The file gets too big, then rebooting will not help. The files need deleted manually.

    ReplyDelete
    Replies
    1. I think you hit the nail on the head here. This is exactly what happened to me, too. I ran updates but couldn't reboot for a couple days and then had this exact problem (cab files being written to windows/temp and huge cbspersist file in windows/logs/cbs).

      Delete
    2. Perfect, bull's eye...! Was long affected with this problem, finally resolved after hitting this page.

      Delete
  5. CAB files are limited to 2GB in size - that's the issue.

    ReplyDelete
  6. I found the problem that was causing the cab creation failure.
    One of my cbspersist_.log files was over 2GB (apparently, someone has been updating, but not rebooting).
    Renaming this file to Not_CbsPersist_.log stopped the rampant cab file creation in the Windows\Temp folder (@ 15 mins for us).

    ReplyDelete
    Replies
    1. Thanks, that's what happened on my Windows 7 SP1 64 bit system. See discussion here http://superuser.com/questions/803842/why-is-cbs-log-file-size-20-gb?lq=1

      Delete
    2. Thanks Jim Harrison. By renaming cbspersist_.log did stopped the cab file creation.

      Delete
  7. Thank you, Felix. Run into a similar problem recently and the large cbspersist_xxxxxxxxxxxx log file is the cause. rename/delete the file stopped the writing of .cab files to the C:\Temp folder.

    In my case, the server is regularly rebooted after applying Windows updates. The file just got larger than usual for some unknown reason(s) this time around, since the server was restarted shortly after applying Windows updates.

    ReplyDelete
  8. Thanks a lot, you helped me out quickly. Just like above, it was a normal scheduled update on SRV2008R2 with normal boot afterwards. Logfilesize was 2,3GB

    ReplyDelete
  9. Thanks for the pointers with this! Helped a lot.

    May I asked how you generated the Disk-space used stats? I'm looking for software to monitor disk space used overtime to calculate growth.

    ReplyDelete
    Replies
    1. As I indicataed at the beginning of the artile, our RMM (Remote Monitoring and Managemnt) software provides stats over time. We used to use Labtech, but have switched to N-Able. Both are commercial packages, costing a few $ per node /month. Spiceworks is a freeware monitoring system. There are a number of dedicated free packages (TreeSize), but they typically don't give you a nice graph.

      Delete
  10. You should use PRTG from Paessler (Germany).
    There is a free version with 100 "sensors" included.

    look at it.

    ReplyDelete
  11. The CBS.perist logs were taking over ALL of my C disk space on my Dell Laptop. Thank you for pointing to a solution!

    ReplyDelete
  12. what are cab_4060_x, cab_5480_x, cab_5556_x, cab_5756_x, cab_5820_x and cab_5304_x that's always store in my lapi local disk(c:)!!! not under any folder.. 20 30 file always seen here i'm delete it many time but they are appears again and again...??

    ReplyDelete
  13. Thank you for this post, this solved my issue with vanishing hard drive space on a test computer I'd spun back up after a long time. It appears that running a significant number of Windows Updates in a short time frame will bloat the CBS logs and create the problem.

    ReplyDelete
  14. What is the fix ? To stop it from happening again.

    ReplyDelete
  15. It seemed to happen on a laptop I worked on for someone due to a number of conditions.
    - For one the battery was no good.
    - Number two it was set to go into STANDBY in a short period, like twenty minutes(it was probably having a hard time finishing updates, etc..).
    - It would probably get unplugged and moved from place to place at times, losing its Standby info thus Windows does not get shut down properly.
    - Now this probably happened at least once during updating, all these times it was probably writing to the error log which may have gotten borked during another standby/unplug etc. They said it kept saying Windows was not shut down properly.

    It was so bad it was all wound up trying to do things it couldn't. That file in the CBS folder was about 2 gig, Windows Update had also gotten borked and had not been able to update for months I noticed.

    So give enough time for updates for one thing. :0) Good luck! Happy New Year!!

    ReplyDelete