Keep search heads as close as possible to indexers. These events can be gathered using a Splunk Universal Forwarder and then forwarded to indexers, which could be located in a central place. Try to collect occurrences that are as close together as possible (in terms of geography and network location). If the events are generated by the same device and have the same format, they are most likely from the same source. Source type can be used to categorize data based on their similarity. Indexes and sourcetypes aid data management. It won’t be easy to adjust these two items afterward. Make indexes and source types a priority. A stand-alone solution is essential for large deployments. The Deployment server is usually co-located with this system. Master Node: Node that serves as the master. Typically, this system serves as the License Master. This system can be used in conjunction with other Splunk services or on its own. Splunk’s Enterprise Security (ES) application is supported via a separate search head.ĭeployment Server: A deployment Server is a server that allows you to deploy software. Any search request will be distributed across all set search-peers, improving search performance. This technique saves search time and provides some data ingest and availability redundancy if a single server fails. Multiple clustered search-peers (indexers) boost data-ingest and search performance. □ĭesign is dependent on the environment but I hope it helps at least get started in the right direction. In the spreadsheet, there’s also a Daily Log Sizing estimator which comes in handy. I’m attaching the spreadsheet for this design. O 10Gb Ethernet NIC, optional 2nd NIC for a management network O 16 CPU cores at 2Ghz or greater speed per core, 48GB RAM O 10Gb Ethernet NIC with optional second NIC O 128GB RAM 32TB disk w/ 900 IOPS (multiple partitions) to accommodate 365-day retention O 2 x 400 GB Solid State Drive, Redundant Array of Independent Disks (RAID) 0 or 1+0 2 Universal Forwarders (Syslog, etc.) aggregation nodes.O 64-bit Red Hat Enterprise Linux distribution O 2 x 300GB, 10,000 RPM SAS hard disks, configured in RAID 1 2 nodes 1 for Deployment Server + 1 for Monitoring Console (DMC), and License Master (LM).The total data set was indexed approximately 6 times quicker when collected by the Universal Forwarder.īelow is what you do NOT want to do when designing the forwarders because it creates a bottleneck:īelow is what you want to do when designing the forwarders:īelow is a design that I’ve done in the past using Microsoft Azure but it’s the same for AWS: The amount of data indexed per second was approximately 3 times higher when collected by a Universal Forwarder. The amount of data sent over the network was approximately 6 times lower with the Universal Forwarder. Performance benefits when choosing Universal Forwarders: The recommendation is to use a Universal Forwarder.īelow you can see a nice comparison between the different forwarders: Below is some info on the differences that came from a Splunk Conference. Universal forwarders are used to collect data from a number of sources and send it to Splunk indexers. One important decision is to decide to use a Universal or Heavy Forwarder. I’m hoping to keep adding to this blog post. There’s a lot you need to think about when designing your Splunk environment.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |