user guide - static.huaweicloud.com tank service user guide issue 05 date 2018-04-17 huawei...

Log Tank Service

User Guide

Issue 08

Date 2018-06-01

HUAWEI TECHNOLOGIES CO., LTD.

Copyright © Huawei Technologies Co., Ltd. 2018. All rights reserved.No part of this document may be reproduced or transmitted in any form or by any means without prior writtenconsent of Huawei Technologies Co., Ltd. Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.All other trademarks and trade names mentioned in this document are the property of their respectiveholders. NoticeThe purchased products, services and features are stipulated by the contract made between Huawei and thecustomer. All or part of the products, services and features described in this document may not be within thepurchase scope or the usage scope. Unless otherwise specified in the contract, all statements, information,and recommendations in this document are provided "AS IS" without warranties, guarantees orrepresentations of any kind, either express or implied.

The information in this document is subject to change without notice. Every effort has been made in thepreparation of this document to ensure accuracy of the contents, but all statements, information, andrecommendations in this document do not constitute a warranty of any kind, express or implied.

Huawei Technologies Co., Ltd.Address: Huawei Industrial Base

Bantian, LonggangShenzhen 518129People's Republic of China

Website: http://www.huawei.com

Email: [email protected]

Issue 08 (2018-06-01) Huawei Proprietary and ConfidentialCopyright © Huawei Technologies Co., Ltd.

i

http://www.huawei.com

mailto:[email protected]

Contents

1 Log Query and Analysis...............................................................................................................11.1 Concepts......................................................................................................................................................................... 11.2 Log Preview and Full-Text Search................................................................................................................................. 21.3 Searching for Logs by Configuring Custom Search.......................................................................................................4

2 Management................................................................................................................................... 72.1 Log Group Management.................................................................................................................................................72.1.1 Creating a Log Group.................................................................................................................................................. 72.1.2 Viewing a Log Group.................................................................................................................................................. 82.1.3 Deleting a Log Group.................................................................................................................................................. 92.2 Log Topic Management................................................................................................................................................102.2.1 Creating a Log Topic................................................................................................................................................. 102.2.2 Viewing a Log Topic..................................................................................................................................................112.2.3 Deleting a Log Topic................................................................................................................................................. 122.3 Log Management..........................................................................................................................................................122.3.1 Viewing Logs.............................................................................................................................................................122.3.2 Viewing Raw Logs.................................................................................................................................................... 132.3.3 Log Preview and Full-Text Search............................................................................................................................ 142.4 Telescope Management................................................................................................................................................ 152.4.1 Modifying Log Collection Settings........................................................................................................................... 162.4.2 Viewing Log Collection Settings...............................................................................................................................192.4.3 Locating Log Collection Errors.................................................................................................................................202.4.4 Disabling Log Collection...........................................................................................................................................212.4.5 Deleting Log Collection Settings.............................................................................................................................. 222.5 Filter Management........................................................................................................................................................232.5.1 Disabling a Filter....................................................................................................................................................... 232.5.2 Deleting a Filter......................................................................................................................................................... 242.6 Log Transfer Management............................................................................................................................................242.6.1 Creating a Log Transfer Task.................................................................................................................................... 252.6.2 Viewing a Log Transfer Task.....................................................................................................................................262.6.3 Modifying a Log Transfer Task................................................................................................................................. 272.6.4 Disabling a Log Transfer Task...................................................................................................................................292.6.5 Deleting a Log Transfer Task.................................................................................................................................... 30

3 FAQs...............................................................................................................................................32

Log Tank ServiceUser Guide Contents


ii

3.1 What Is Log Tank Service?...........................................................................................................................................323.2 What Are the Restrictions that Apply to the Use of LTS?............................................................................................323.3 What Kind of Logs Can LTS Collect?..........................................................................................................................323.4 How Does LTS Collect Logs?...................................................................................................................................... 323.5 What Are LTS Fees?.....................................................................................................................................................323.6 What Is Telescope?....................................................................................................................................................... 333.7 Whether Telescope Supports the Collection of Historical Logs?.................................................................................333.8 What Restrictions Are on Log Queries?.......................................................................................................................333.9 How Can I Locate Faults Using Logs?.........................................................................................................................333.10 What OSs Are Supported by Telescope?....................................................................................................................333.11 How Do I Restart Telescope?..................................................................................................................................... 343.12 What Should I Do if the Agent Heartbeat Status Displays "No Agent heartbeat messages have been receivedduring the last 3 minutes"?................................................................................................................................................. 34

A Change History........................................................................................................................... 36

Log Tank ServiceUser Guide Contents


iii

1 Log Query and Analysis

LTS enables you to preview and query logs. It provides second-level query and fault locatingcapabilities, helping you improve O&M and operation efficiency. With LTS, you can easilycope with common O&M issues, such as troubleshooting server faults, application bugtracking and analysis, and service data analysis.

1.1 Concepts

Log PreviewLog preview enables you to quickly read log data. However, this function does not supportlog querying by keyword, and logs cannot be viewed on the Dashboard page.

NOTE

Log data on the Dashboard page can be refreshed in real time only after you enable the log searchfunction.

Before querying and analyzing logs, you need to ensure that you can query logs in real timeand check whether logs are uploaded then enable the log search function.

Search ModeLTS supports various log search modes, including full-text search and custom search.

l Full-Text Search: indicates that LTS creates indexes for all fields in logs. In this searchmode, you can search for any field in logs.For example, the account information /var/log/secure must be recorded upon any login.[root@www~]# cat /var/log/secure

Log Tank ServiceUser Guide 1 Log Query and Analysis


1

1 Mar 14 15:33:00 www atd[18701] : pam_unix (atd:session): session opened for userroot by (uid=0)

2 Mar 14 15:33:00 www atd[18701] : pam_unix (atd:session): session closed for userroot

3 Mar 14 16:01:51 www su : pam_unix(su-1:auth):authentication failure; logname=vbird uid= 500 euid= 0 tty= pts/1 ruser=vbird rhost= user=root

4 Mar 14 16:01:51 www su : pam_unix(su-1:session): session opened for user root byvbird(uid=500)

5 Mar 14 16:01:51 www su : pam_unix(su-1:session): session closed for user root

With full-text search, you can search for logs as easy as using an engine. You only needto enter the required text in the search box, for example, enter authentication for theabove log. Then, the following log is matched:


In addition, you can query the above log using one of its fields. For example, the log canalso be matched after you enter rhost= user=root.


Full-text search is easy to operate. After enabling this function, you can search for full-text logs without any settings, especially those logs with complex nested structures,helping you quickly locate abnormal logs.

l Custom Search: If your log format is fixed and you prefer a certain search mode, youcan customize your own search way, for example, select a delimiter to separate logfields. Then, LTS segments your logs in the way you set so that you can view or searchfor logs in the key-value format.

LTS enables you to select common invisible characters as delimiters, such as spaces andline feeds, or enter 1 to 3 characters based on your logs.

For example, if you need to query the following log:

pam_unix(su-1:auth)

you can select the delimiter mode and enter an underscore (_) as your delimiter.Therefore, your raw log is segmented into pam and unix(su-1:auth).

1.2 Log Preview and Full-Text Search

Scenarios

Use keywords to search for logs that have been uploaded to the LTS console.

Prerequisitesl You have obtained a username and password for logging in to the management console.

l You have created a log group.

l You have created a log topic.

l You have collected logs.



2

Procedure

1. Log in to the management console,click in the upper left corner, and select thedesired region and project.

2. Choose Management & Deployment > Log Tank Service.The LTS console is displayed.

3. In the navigation pane on the left, choose Log Management.The Log Management page is displayed.

4. Click the name of the log group that contains the target logs.The log topic list is displayed.

5. Click Search Log in the row where the log group is located, or click the name of the logtopic to which the log belongs.The log list is displayed.The log preview page helps you determine whether logs are properly collected and vieworiginal logs. The log preview function allows you to view only the latest 1000 logs.

6. Click Configure Search in the upper right corner of the page. On the displayed page,enable the full-text search function. After the search function is enabled, you can viewfull logs on the log preview page, and a log search box is displayed in the upper part ofthe page.

7. Click and specify the time range of the target logs.

Figure 1-1 Selecting the time range

NOTICEl The time range cannot exceed seven days.l Search results display a maximum of 1,000,000 of the latest logs within the time

range you selected. If the number of logs exceeds the maximum limit, you need toreset the time range.



3

8. Enter a keyword in the search box.

Figure 1-2 Entering a keyword

1.3 Searching for Logs by Configuring Custom Search

ScenariosThis section describes how to search for logs using custom search if your logs are in fixedformat, making it easy for you to view logs in the key-value format.

Procedure


2. Click Service List and choose Management & Deployment > Log Tank Service.3. In the navigation pane on the left, choose Log Management.4. On the Log Management page, click the name of the log group where the target log

belongs.5. Click Search log in the row where the log group is located. The log preview page is

displayed.6. Click Configure Search in the upper right corner. The Configure Search page is

displayed.7. Select Custom Search for Log Search Mode. You can use a delimiter to extract key

fields of your logs, making it easy to query and analyze logs.8. Select a delimiter. You can select Space or Line Feed or enter 1 to 3 characters as a

delimiter based on your logs.

NOTE

Custom delimiters do not support path, ip, topic_id, time, collect_time, message, message_size,host_name, and original as keys.

9. Log Example enables you to check whether your log can be parsed using your customsearch conditions and to verify the parsing result display. Therefore, you are advised topaste a real log that needs parsing.



4

10. Extraction Result displays the parsing result of the entered log example based on yoursearch conditions.If you want to display the parsed log in the key-value format, you are advised to set keysfor the values searched to identify different extract results.

11. Retain Original Log: After you enable Retain Original Log, the fields to which youhave assigned keys will be displayed in the key-value format, and those without keyswill be discarded. Therefore, you are advised to enable Retain Original Log.

The following log is used as an example to describe how to customize log search using adelimiter.

2018/05/28 00:00:10 GMT+08:00,200,tag,getResourcesInsByTags,ConsoleAction,

Use this log as an example. Select Custom Search for Log Search Mode and Custom forDelimiter. Select the comma (,) as a delimiter. Paste this log at the log example area and clickText.

The extraction results are shown in Figure 1-3.

Figure 1-3 Log separating results

For fields you need to keep, name them in the Key column. Non-named fields are deemed tobe useless for you. You can choose to retain the original log. Therefore, the fields to whichyou assign keys are displayed in the key-value format, and those fields without keys are



5

displayed in the raw log format. You can determine whether to retain the parsed fields basedon your own service requirements.

For the extraction of this log example, we assume that all values need to be preserved, so weneed to assign keys for all values.

Figure 1-4 Naming the extracted values

After completing configuration, click OK to view logs displayed in the key-value format.



6

2 Management

2.1 Log Group Management

2.1.1 Creating a Log Group

ScenariosCreate a log group for log storage and query. Each cloud account can create a maximum of 10log groups.

PrerequisitesYou have applied for the LTS service and obtained your username and password for loggingin to the management console.

Procedure


2. Under Management & Deployment, click Log Tank Service.The LTS console is displayed.


4. Click Create Log Group.The Create Log Group dialog box is displayed.

5. Enter a log group name.

Log Tank ServiceUser Guide 2 Management


7

Figure 2-1 Create Log Group

Table 2-1 Parameters for creating a log group

Parameter Description ExampleValue

Log GroupName

Specifies the log group name, which:l Can only consist of Chinese characters, upper and

lower case letters, digits, underscores (_), orhyphens (-).

l Must be a string of 1 to 64 characters.l Must be unique.

LogGroup 1

LogRetentionDuration

The default retention period for log data is 7 days.Any logs exceeding the expiration period becomeinvalid.

N/A

6. Click OK.

2.1.2 Viewing a Log Group

ScenariosView information about existing log groups, including the log group name, ID, expirationdate, and creation time.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.

Procedure




8

2. Under Management & Deployment, click Log Tank Service.

The LTS console is displayed.

3. In the navigation pane on the left, choose Log Management.

The Log Management page is displayed.

4. View log group information in the log group list.

Figure 2-2 Viewing log group information

2.1.3 Deleting a Log Group

Scenarios

Delete a log group that will no longer be used on the LTS console.

NOTE

Ensure that no log topics and log transfer tasks are under the target log group. Deleted log groups cannotbe restored. Therefore, you need to exercise caution when performing this deletion operation.

Prerequisitesl You have obtained a username and password for logging in to the management console.

l You have deleted all log topics under the target log group.

l You have deleted all log transfer tasks associated with the log group.

Procedure


2. Under Management & Deployment, click Log Tank Service.



The Log Management page is displayed.

4. In the log group list, locate the row containing the target log group and click Delete.

The Delete dialog box is displayed.



9

Figure 2-3 Deleting a log group

5. Click OK.

2.2 Log Topic Management

2.2.1 Creating a Log Topic

ScenariosCreate a log topic under the created log group on the LTS console. View and query raw logsunder that log topic.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.

Procedure




4. On the Log Management page, click the name of the newly created log group. Abovethe log group list, click Create Log Topic.

5. In the displayed dialog box, enter the log topic name, for example, LogTopic 1.



10

Table 2-2 Parameters for creating a log topic

Parameter Description

Log Topic Name Specifies the log topic name, which:l Can only consist of Chinese

characters, upper and lower caseletters, digits, underscores (_), orhyphens (-).

l Must be a string of 1 to 64 characters.l Must be unique within a log group.

6. Click OK.

A new log topic is created.

2.2.2 Viewing a Log Topic

ScenariosView information about existing log topics, including the log topic name, ID, log generatedtime, and creation time.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.l You have created a log topic.

Procedure


2. Under Management & Deployment, click Log Tank Service.The Log Tank Service page is displayed.


4. On the Log Management page, click the log group where the log topic is located. In thelog topic list, view the log topic information.

Figure 2-4 Viewing log topic information



11

2.2.3 Deleting a Log Topic

ScenariosDelete a log topic that will no longer be used.

NOTE

Ensure that no log transfer tasks are under the target log topic. Deleted log groups cannot be restored.Therefore, you need to exercise caution when performing a deletion operation.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have deleted the log transfer task associated with the target log topic.

Procedure




4. In the log group list, click the log group where the target log topic is located.5. In the log topic list, locate the row that contains the target log topic and click Delete.

The Delete dialog box is displayed.

Figure 2-5 Deleting a log topic

6. Click OK.

2.3 Log Management

2.3.1 Viewing Logs

ScenariosView logs on the LTS console.



12

Prerequisitesl You have obtained the username and password for logging in to the management

console.l You have created a log group.l You have created a log topic.l You have collected logs.

Procedure




4. Click the name of the log group to which logs to be viewed belong.The log topic list is displayed.

5. Click the name of the log topic to which logs to be viewed belong.In the log topic list, view log information.

Figure 2-6 Viewing log information

2.3.2 Viewing Raw Logs

Scenarios

View raw logs on the LTS console.

Prerequisitesl You have obtained the username and password for logging in to the management

console.l You have created a log group.l You have created a log topic.l You have collected logs.

Procedure






13

The Log Management page is displayed.4. Click the name of the log group to which logs to be viewed belong.

The log topic list is displayed.5. Click the name of the log topic to which logs to be viewed belong.

The log list is displayed.6. In the log list, locate the target log click View Raw Log.

View the message displayed.

Figure 2-7 Log list

7. Click Close.

Figure 2-8 Viewing a raw log

2.3.3 Log Preview and Full-Text Search

Scenarios

Use keywords to search for logs that have been uploaded to the LTS console.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.l You have created a log topic.l You have collected logs.

Procedure




4. Click the name of the log group that contains the target logs.The log topic list is displayed.



14

5. Click Search Log in the row where the log group is located, or click the name of the logtopic to which the log belongs.

The log list is displayed.

The log preview page helps you determine whether logs are properly collected and vieworiginal logs. The log preview function allows you to view only the latest 1000 logs.

6. Click Configure Search in the upper right corner of the page. On the displayed page,enable the full-text search function. After the search function is enabled, you can viewfull logs on the log preview page, and a log search box is displayed in the upper part ofthe page.

7. Click and specify the time range of the target logs.

Figure 2-9 Selecting the time range

NOTICEl The time range cannot exceed seven days.

l Search results display a maximum of 1,000,000 of the latest logs within the timerange you selected. If the number of logs exceeds the maximum limit, you need toreset the time range.

8. Enter a keyword in the search box.

Figure 2-10 Entering a keyword

2.4 Telescope Management



15

2.4.1 Modifying Log Collection Settings

Scenarios

This section describes how to modify log collection settings when the log format changes, orwhen you want to modify a log group and log topic.

Prerequisitesl You have created a log group.

l You have created a log topic.

l You have installed Telescope by referring to section "Installing Telescope" in the QuickStart.

l You have completed the Telescope configuration by referring to section "ConfiguringCollection Settings" in the Quick Start.

Procedure


2. Choose Management & Deployment > Log Tank Service.


3. In the navigation pane on the left, click Agent Management.

The Agent Management page is displayed.

4. In the Agent list, locate the Agent for which the configurations need to be modified andclick Modify.

NOTE

The modification only applies to the instance where Agent is located.

5. On the Configure Agent page, configure log collection information by referring toTable 2-3.

Figure 2-11 Modifying Agent



16

Table 2-3 Parameter description


ConfigurationName

Indicates the name of the configuration to be modified.

Instance Name Indicates the name of the instance to be modified.

Start LogCollection

Specifies whether to enable the log collection function.When you configure the log collection information, the logcollection function will be enabled by default. You can disablethe function as needed.

6. Click Next.7. On the Configure Collection Settings page, modify the log collection information by

referring to Table 2-4.

Table 2-4 Parameters to be configured


Log Format Select a log format based on the logs you need to collect.If each line in a log file represents a datum, select Loggeneration time. If the log contains Java exceptions (a logdatum is displayed in several lines), select By regularexpression.l Single-linel Multi-line

NOTEOnly the logs in the directory that you specified can be collected toa single log group. If the logs you need to collect are stored indifferent directories, click Configure Collection Settings to add anew directory.

Log Group Name Specifies the log group name. Select a created log group. Logswill be collected and placed in this log group.

Log Topic Name Specifies the log topic. Select a created log topic. Logs will becollected under this log topic in the log group that youspecified.



17


Path Enter the directory where the logs to be collected are stored.The parameter can be a relative path or an absolute path, andwildcards are supported.The following are two examples for Linux and Windows,respectively:Linux OS paths:/var/log/db/*.log/var/log/db/db.log*/var/log/db/db.*.log*../../log/*.log* Windows OS path:D:\agentNOTE

l The system collects logs only in the root directory. For example, ifthe /var/log/db directory has subdirectory db2, logs in db2 cannotbe collected currently.

l Check whether your sensitive information is in the collectingscope.

l The log collection path cannot be /usr/local/telescope/log.

Log Time System Time: Specifies the log parsing time by default. Thistime mode is suitable for logs without the printing time.

Time Wildcard: Specifies the log time in the form ofwildcards. If the collected log data contains time, use thisparameter to mark a piece of log data. For example, if the logtime is 2017-01-01 23:59:59, the wildcard is YYYY-MM-DDhh:mm:ss.NOTE

If a log does not contain year information, Agent automaticallysupplements the current year data as the year data during collection.

The following are two examples for Linux and Windows,respectively:M - month (1) MM - month (01) MMM - month (Jan) MMMM - month (January)D - day (2) DD - day (02) DDD - day (Mon) DDDD - day (Monday) YY - year (06) YYYY - year (2006) hh - hours (15) mm - minutes (04) ss - seconds (05) hpm - hours (03PM)h:mmpm - hours:minutes (03:04PM)h:mm:sspm - hours:minutes:seconds (03:04:05PM) hh:mm:ss ZZZZ (16:05:06 +0100) hh:mm:ss ZZZ (16:05:06 CET) hh:mm:ss ZZ (16:05:06 +01:00)



18


Log Segmentation If each line in a log file represents a datum, select Loggeneration time. If the log contains Java exceptions (a logdatum is displayed in several lines), select By regularexpression.l Log generation timel Regular expression

NOTE

l Only configure this parameter when the format of logs youneed to collect is multi-lined.

l If you select Log generation time, multi-line logs are dividedby time wildcard. If you select By regular expression, multi-line logs are divided by regular expression.

Regular Expression This regular expression is used to mark a piece of log data.NOTE

Only configure this parameter when you select Multi-line for LogFormat and By regular expression for Log Segmentation.

8. Click Submit.

2.4.2 Viewing Log Collection Settings

Scenarios

This section describes how to view Agent configuration details.

Prerequisitesl You have created a log group.l You have created a log topic.l You have installed Telescope by referring to section "Installing Telescope" in the Quick

Start.l You have completed the Telescope configuration by referring to section "Configuring

Collection Settings" in the Quick Start.

Procedure



3. In the navigation pane on the left, click Agent Management.The Agent Management page is displayed.

4. In the Agent list, locate the row containing the target Agent and click .For details, see Figure 2-12.



19

Figure 2-12 Viewing log collection settings

NOTE

If the VM used for installing Agent is deleted, the instance name becomes "--". Whether to deletethe Agent required by the VM depends on yourself.

5. After the check is complete, click .

2.4.3 Locating Log Collection Errors

ScenariosErrors may occur during log collection. This section describes how to check the errorinformation.


Start.l You have completed the Telescope configuration by referring to section "Configuring

Collection Settings" in the Quick Start.

Procedure




4. In the Agent list, check whether the heartbeat status of the target Agent is normal.– If the heartbeat status is normal, go to 5.– If the heartbeat status is abnormal, click Abnormal in the Heartbeat Status

column.View the displayed Abnormal heartbeat status dialogue box and then click Close.

– If the heartbeat status is abnormal, click Abnormal in the Heartbeat Statuscolumn. The Abnormal heartbeat status dialog box is displayed. If the message"No Agent heartbeat messages have been received during the last 3 minutes" isdisplayed, rectify the fault by based on section "What Should I Do if the AgentHeartbeat Status Displays "No Agent heartbeat messages have been received duringthe last 3 minutes"?"



20

5. Locate the row that contains the abnormal Agent and click Diagnose in the Operationcolumn.The Diagnose page is displayed.

Figure 2-13 Viewing the error information

NOTE

l If the heartbeat status is Abnormal, the Agent may be uninstalled or the Agent process iskilled. In this case, the diagnose log cannot be collected and is empty.

l Currently, errors that occurred during the last 10 minutes will be displayed.

6. On the Diagnose page, view the error details.

Figure 2-14 Error information

7. Click Close.

2.4.4 Disabling Log Collection

ScenariosThis section describes how to disable the log collection function.


Start.



21


Procedure




4. In the Agent list, locate the row containing the target Agent and click Modify.5. On the displayed page, disable Start Log Collection.

Figure 2-15 Disabling log collection

NOTICEThis modification takes effect immediately. Exercise caution when enabling or disablingthis function.

6. Click Next.7. Click Submit.

2.4.5 Deleting Log Collection Settings

Scenarios

This section describes how to delete log collection settings on the LTS console if you havealready deleted Agent from an instance.


Start.



22


Procedure




4. In the Agent list, locate the row containing the Agent configurations that you want todelete and click Delete.The Delete dialog box is displayed.

5. Click OK.

NOTE

After you delete these configurations, the log collections function will automatically be disabled.However, the Agent installed on the instance will not be deleted.

2.5 Filter Management

2.5.1 Disabling a Filter

Scenarios

Current filtering keywords become invalid if the format or content of logs under the log topic(for which one or more filters have been configured) changes. This section describes how todisable an invalid filter or a filter that you do not need to monitor or send alarms for usingCES.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.l You have created a log topic.l You have collected logs.l You have configured a custom metric filter based on section Configuring a Custom

Metric Filter" in the Quick Start.

Procedure

1. Switch to the console home, click in the upper left corner of the managementconsole, and select the desired region and project.




23


4. In the log group list, click the name of the log group where the target filter is located.The log topic list is displayed.

5. Click the expanding button in front of the target log topic. The filter list is displayed.6. Click Disable in the row where the target filter is located.

NOTE

After the filter is disabled, you cannot monitor the metric preset in the filter on CES. In addition,the previously configured alarm rules may trigger an alarm due to insufficient data.

2.5.2 Deleting a Filter

Scenarios

A maximum of five key metric filters can be configured under a log topic. If you want to adda filter, you need to delete unnecessary filters and try again. This section describes how todelete the unnecessary filters from the log list.

Prerequisitesl You have obtained a username and password for logging in to the management console.l You have created a log group.l You have created a log topic.l You have collected logs.l You have configured a custom metric filter based on section Configuring a Custom

Metric Filter" in the Quick Start.

Procedure

1. Switch to the console home, click in the upper left corner of the managementconsole, and select the desired region and project.



4. In the log group list, click the name of the log group where the target filter is located.The log topic list is displayed.

5. Click the expanding button in front of the target log topic. The filter list is displayed.6. Click Delete in the row where the target filter is located.

NOTE

After the filter is deleted, you cannot monitor the metric preset in the filter on CES. In addition,the previously configured alarm rules may trigger an alarm due to insufficient data.

2.6 Log Transfer Management



24

2.6.1 Creating a Log Transfer Task

ScenariosLogs uploaded to the LTS platform can be stored for 7 days. For long-term storage, transferlogs to OBS buckets.

NOTICEStorage fees depend on how much and how long you store.

Prerequisitesl You have created a log group.

l You have created a log topic.l You have uploaded logs.l You have created OBS buckets.

Procedure

1. Log in to the management console, click in the upper left corner, and select thedesired region and project.

2. Choose Management & Deployment > Log Tank Service.The Log Tank Service page is displayed.

3. In the navigation pane on the left, choose Log Transfer.The Log Transfer page is displayed.

4. Click Configure Log Transfer.The Configure Log Transfer page is displayed.

5. Select the target log group, log topic, and OBS bucket and enter the prefix of the logs tobe transferred.



25

Figure 2-16 Configuring log transfer

Table 2-5 Parameters for configuring log transfer


Log Group Name Specifies the log group name. Select a created log group.

Log Topic Specifies the log topic. Select a created log topic.

OBS Bucket Specifies the name of the created OBS bucket.NOTE

You can create an OBS bucket or reconfigure the OBS bucket.For detailed operations, see section Managing Buckets in theObject Storage Service Console Operation Guide.

Log Prefix Enables you to distinguish log files from other files in anOBS bucket.

Transfer Log to OBS Allows you to enable log transfer.

Log Transfer Interval Specifies the interval at which logs are automaticallytransferred to the OBS bucket. The default value is 300s.

6. Click OK.

2.6.2 Viewing a Log Transfer Task

ScenariosThis section describes how to view a log transfer task that has been configured.



26

https://support.huaweicloud.com/en-us/usermanual-obs/en-us_topic_0045853506.html

Prerequisitesl You have created a log group.l You have created a log topic.l You have uploaded logs.l You have created OBS buckets.l You have configured log transfer tasks.

Procedure




4. View information of a log transfer task, including the log group name, log topic, OBSbucket, log prefix, transfer task status, and creation time.

Figure 2-17 Viewing a log transfer task

2.6.3 Modifying a Log Transfer Task

Scenarios

This section describes how to modify a configured log transfer task. You can also modify yourlog topic, OBS bucket, log prefix, and transfer task status.


Procedure


2. Choose Management & Deployment > Log Tank Service.The Log Tank Service page is displayed.

3. In the navigation pane on the left, choose Log Transfer.



27

The Log Transfer page is displayed.4. Locate the target log group and click Modify.

The Modify page is displayed.

Figure 2-18 Modifying a log transfer task

5. Modify log transfer task information.


Table 2-6 Parameters for modifying a log transfer task


Log Group Name Specifies the log group name. Select a created log group.

Log Topic Specifies the log topic. Select a created log topic.

OBS Bucket Specifies the name of the created OBS bucket.

Log Prefix Enables you to distinguish log files from other files in anOBS bucket.

Transfer Log to OBS Allows you to enable log transfer.

Log Transfer Interval Specifies the interval at which logs are automaticallytransferred to the OBS bucket. The default value is 300s.



28

6. Click OK.

2.6.4 Disabling a Log Transfer Task

ScenariosThis section describes how to disable a log transfer task that has been configured. After thetask is disabled, related logs cannot be transferred to OBS buckets.

NOTE

After a task is disabled, related logs cannot be transferred to OBS buckets. The task can be enabledagain when you need it.


Procedure




4. In the displayed log group list, locate the target log group and click Modify.The Modify page is displayed.


5. Disable Transfer Log to OBS.



29

Figure 2-21 Disabling a log transfer task

6. Click OK.

2.6.5 Deleting a Log Transfer Task

Scenarios

This section describes how to delete a log transfer task that has been configured. After a logtransfer task is deleted, logs will no longer be automatically transferred to the specified OBSbucket. Deleted log transfer settings cannot be restored.

NOTICEBefore deleting a log transfer task, ensure that it has been disabled.


Procedure




30



4. In the log group list, locate the row containing the target log group and click Delete.The Delete dialog box is displayed.

5. Click OK.



31

3 FAQs

3.1 What Is Log Tank Service?Log Tank Service (LTS) collects, analyzes, and stores log data. It helps you improve both logprocessing and routine O&M efficiency.

3.2 What Are the Restrictions that Apply to the Use ofLTS?

l Each cloud account can create a maximum of 10 log groups.l You can create a maximum of 10 log topics under a log group.l You can store log data on the LTS console free of charge for 7 days.l The maximum size of a log is 127 KB, and any content exceeding the limit will be

removed.

3.3 What Kind of Logs Can LTS Collect?Currently, LTS only supports log file collection, and the logs (UTC/GMT+08:00) must befrom the last seven days.

3.4 How Does LTS Collect Logs?Linux Telescope is provided to collect real-time updated logs from disk files in different OSs.

3.5 What Are LTS Fees?During the beta test LTS is free of charge. However, transferring logs to OBS buckets entailscharges in accordance with the charging standards of OBS.

Log Tank ServiceUser Guide 3 FAQs


32

3.6 What Is Telescope?Telescope is a client agent provided by HUAWEI CLOUD. It helps you collect ECSmonitoring metrics and log data without installation. The log service module of Telescopelistens log files in a specified directory and automatically uploads updated logs to a specifiedlog group and topic that you set.

3.7 Whether Telescope Supports the Collection ofHistorical Logs?

Telescope can only collect data generated in the last seven days.

3.8 What Restrictions Are on Log Queries?Currently, you can only use complete fields in queries.

3.9 How Can I Locate Faults Using Logs?1. Logs are automatically collected if information similar to the following is displayed in

the lts.log file under the /usr/local/telescope/log directory.2017-08-31/11:40:33 [INF] [data.go:17] Start to extract...

2. To locate faults easily during log collection, you can modify the log level in the bin/logs_config.xml file to control the level of logs exported from Agent.

3. The default value of <seelog minlevel="info"> is info. You can change this value todebug, warn, or error as required. After modifying this parameter, you need to restartthe service before the modification takes effect.

3.10 What OSs Are Supported by Telescope?The following table lists OSs compatible with Telescope. OSs not included in the table arebeing tested.

NOTICEUsing unverified OSs or versions may adversely affect your services. Exercise caution whenchoosing your OS.

Table 3-1 lists verified OSs.

Table 3-1 System version

Available OS (64 bit) Version

CentOS 6.5, 7.1



33

Available OS (64 bit) Version

openSUSE 13.2

Debian 7.5

Ubuntu 14.04

EulerOS 2.2

3.11 How Do I Restart Telescope?Generally, there are two Telescope processes: parent process and child process. If an erroroccurs during log collection, perform the following steps:

1. Run the following command to check the Telescope process:ps -aux|grep telescope– If the process is not present, run the following command to restart Telescope:

service telescoped restart– If the process is present, run the following commands to restart Telescope:

kill -9 Process IDservice telescoped restart

3.12 What Should I Do if the Agent Heartbeat StatusDisplays "No Agent heartbeat messages have beenreceived during the last 3 minutes"?

In the Agent list, check the heartbeat status of the target Agent. If the status is abnormal, clickAbnormal in the Heartbeat Status column. If a message indicating "No Agent heartbeatmessages have been received during the last 3 minutes" is displayed, rectify the fault byperforming the following operations:

1. In the Agent Management list, find the instance corresponding to the abnormal Agent.2. Click Service List and choose Computing > Elastic Cloud Server. Then on the ECS

console, locate the row containing the target ECS and click Remote Login.

3. After the login is successful, run the following commands to check whether the networkis faulty:



34

cd /usr/local/telescope/bincat conf_ces.jsonIn the command output, check the endpoint address of the region where the current ECSresides.Run the following command:ping Endpoint address of the region where the current ECS resides– If the command output is normal, rectify the network fault.– If it is abnormal, contact technical support.



35

A Change History

Release Date What's New

2018-05-31 This issue is the eighth official release,which incorporates the following changes:l Supported real-time query of collected

logs.l Added the custom log search function.l Added a feature that Agent automatically

supplements the year information duringlog collection.

l Supported system log collection.

2018-05-26 This is the seventh official release, whichincorporates the following changes:l Supported real-time query of collected

logs.l Added the feature of supporting Agent

on Windows ECSs.l Added a feature that Agent automatically


2018-04-23 This issue is the sixth official release.l Added the log preview function in

section "Searching for Logs".l Added the feature of supporting Agent

on Windows ECSs.l Added a feature that Agent automatically


Log Tank ServiceUser Guide A Change History


36

Release Date What's New

2018-04-10 This issue is the fifth official release, whichincorporates the following change:l Added the function of filtering custom

metrics.l Interconnected with CES and enabled

the function of monitoring and sendingan alarm for the number of keywords inlogs.

l Enabled the function of configuring theAK and SK automatically.

2018-02-06 This issue is the fourth official release,which incorporates the following changes:Added the relationship with CTS in section"Related Services".

2017-12-21 This issue is the third official release.Added Section "Installing TelescopeAutomatically With Cloud-Init".

2017-11-22 This issue is the second official release,which incorporates the following change:l Optimized the flowchart for using LTS.l Modified the content of section

"Configuring Log Collection Settings".l Added the Agent management function.

2017-09-20 This issue is the first official release.

Log Tank ServiceUser Guide A Change History


37

user guide - static.huaweicloud.com tank service user guide issue 05 date 2018-04-17 huawei...

Documents