After installing the FLUME, need to configure the Java property file. The Java properties file having the key value pairs and hierarchy property settings. The below are the tasks to complete in configuration file –

1. Defining the flow
2. Configuring Sources
3. Configuring Sinks
4. Configuring Channels
5. Set the channel for source
6. Set the channel for sink

Normally, multiple agents are available in Flume and those agents can be configured by specifying their name.

Defining the Flow -

There are three different components available. i.e. Sources, sinks and channels. Those needs to be named to get them configured.

The components configuration can be shown below.

<Agent_name>.sources = <Source-name>
<Agent_name>.sinks = <Sink-name>
<Agent_name>.channels = <Channel1-name>
 <Channel2-name>…….<Channeln-name>

The FLUME supports various Sources, Sinks and channels and can use any combination of source, sinks and channels.

The list of sources sinks and channels are shown in below table.

Configuring sources -

After defining the flow, need to set properties of each source. The configuring sources can be done in hierarchical namespace fashion. The component type set first and other values for the properties specific to each component will set next.

The sources configuration can be shown below.

# properties for sources
<Agent_name>.sources.<Source_name>.type = <someValue>
<Agent_name>.sources.<Source_name>.<someProperty> = <someValue>
<Agent_name>.sources.<Source_name>.<someProperty> = <someValue>
………
<Agent_name>.sources.<Source_name>.<someProperty> = <someValue>

Configuring sinks -

After defining the flow, need to set properties of each sink. The configuring sinks can be done in hierarchical namespace fashion. The component type set first and other values for the properties specific to each component will set next.

The sinks configuration can be shown below.

# properties for sinks
<Agent_name>.sinks.<Sink_name>.type = <someValue>
<Agent_name>.sinks.<Sink_name>.<someProperty> = <someValue>
<Agent_name>.sinks.<Sink_name>.<someProperty> = <someValue>
………
<Agent_name>.sinks.<Sink_name>.<someProperty> = <someValue>

Configuring Channel -

After defining the flow, need to set properties of each channel. The configuring channel can be done in hierarchical namespace fashion. The component type set first and other values for the properties specific to each component will set next.

The channel configuration can be shown below.

# properties for channels
<Agent_name>.channel.<Channel_name>.type = <someValue>
<Agent_name>.channel.<Channel_name>.<someProperty> = <someValue>
<Agent_name>.channel.<Channel_name>.<someProperty> = <someValue>
………
<Agent_name>.channel.<Channel_name>.<someProperty> = <someValue>

Set the channel for source -

Normally Channel connects with sources. So it is mandatory to set a source to a particular channel to identify which source needs to use for connection establishment.

The Source setting to channel can be shown below.

# set channel for source
<Agent_name>.sources.<Source_name>.channels = <Channel1> <Channel2>
		... <Channel-n>

Set the channel for sink -

Normally Channel connects with sink. So it is mandatory to set a source to a particular channel to identify which sink needs to use for connection establishment.

The Sink setting to channel can be shown below.

# set channel for source
<Agent_name>.sinks.<Sink_name>.channel = <Channel1> <Channel2>
		... <Channel-n>

After configuration, Agent which is configured needs to start up.

Starting Flume Agent -

The starting of Flume agent can be done in the following way.

$ bin/flume-ng agent -n $agent_name -c conf -f 
		conf/flume-conf.properties.template

Now the agent will start running source and sinks configured in the given properties file.