Hi, I've been using TS with more than 5M records ...
# community-help
v
Hi, I've been using TS with more than 5M records for over a year without issues (and it continues to do so). The thing is, I just checked the disk with
smartctl
and it shows a very high amount of writes (11.6 PB in total). How can I reduce disk writes? I thought that since TS is an in-memory DB, it shouldn’t generate so many disk writes. Also, the
pidstat -dl 20
command reports around 213675.16 kB_wr/s (it varies, but you get the idea). Currently, I'm syncing about 3k items every 5 seconds. ---- === START OF SMART DATA SECTION === SMART overall-health self-assessment test result: FAILED! - NVM subsystem reliability has been degraded SMART/Health Information (NVMe Log 0x02) Critical Warning: 0x04 Temperature: 48 Celsius Available Spare: 100% Available Spare Threshold: 10% Percentage Used: 140% Data Units Read: 3,298,109,870 [1.68 PB] Data Units Written: 22,768,223,925 [11.6 PB] Host Read Commands: 15,870,470,592 Host Write Commands: 22,956,431,236 Controller Busy Time: 196,312 Power Cycles: 22 Power On Hours: 22,456 Unsafe Shutdowns: 7 Media and Data Integrity Errors: 0 Error Information Log Entries: 0 Warning Comp. Temperature Time: 0 Critical Comp. Temperature Time: 0 Temperature Sensor 1: 48 Celsius Temperature Sensor 2: 59 Celsius Error Information (NVMe Log 0x01, 16 of 64 entries) No Errors Logged
a
Hey Victor, Despite being a in-memory DB, it will still write all your data to disk, since this is how it will persist your data on off times. On top of data, it will also write all your data to disk on snapshots for example. So you can think of a few GBs being written on every snapshot triggered.
v
Sure, the snapshot is being executed every 3600 seconds (default value), but it keeps writing data constantly in between. I read somewhere that TS uses RocksDB — would changing some of those parameters help reduce disk writes? https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning
a
cc: @Kishore Nallan