Hello everyone,
Over a ~26 hour period I am seeing several fatal errors regarding the typing thread. I've done some research and there is very little information about threads to be found online in regards to how splunk uses them. I am making the assumption that the typing thread is the thread in charge of the typing queue.
The actual error is found in the splunk crash log:
Fatal thread error: pthread_mutex_lock: Invalid argument: 78 threads active in typing
(followed by several more with varying number of active threads, all less than 78)
Dying on fatal thread error 22 from pthread_mutex_lock, thread typing
As far as I remember, mutex are usedto control the availability of threads on a system to prevent race conditions in multithreading. This leads me to believe that perhaps the system is running out of threads when trying to use the typing queue. If this is true, a few things confuse me:
1. If this is running out of threads, why would the active thread count fluctuate on the fatal attempt?
2. If this is the case, why would splunk not have a graceful way of handling this?
These questions I have with my conclusion lead me to believe there is something I'm missing. (My knowledge of mutex is from years ago, so I could very well be wrong).
Any insight would be great. The issue hasn't resurfaced in ~20 days and has only happened once, but knowing the cause would help prevent it in the future.
Thank you.
↧