Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

DSpace 1.6 and newer versions uses the Apache Solr SOLR application underlying the statistics. There is no need to download any separate softwareSOLR enables performant searching and adding to vast amounts of (usage) data.
Unlike previous versions, enabling statistics in DSpace does not require additional installation or customization. All the necessary software is included.

Table of Contents
minLevel2
outlinetrue
stylenone

What is exactly being logged ?

Each time a page or file gets requested, this request is being logged. The logging happens at the server side, and doesn't require a javascript like Google Analytics does, to provide usage data.

Definition of which fields are to be stored happens in the file dspace/solr/statistics/conf/schema.xml.
Some example fields, that can be stored per usage event, include:

Code Block

<field name="type" type="integer" indexed="true" stored="true" required="true" /> 
<field name="id" type="integer" indexed="true" stored="true" required="true" />
<field name="ip" type="string" indexed="true" stored="true" required="false" /> 
<field name="time" type="date" indexed="true" stored="true" required="true" />
<field name="epersonid" type="integer" indexed="true" stored="true" required="false" /> 
<field name="country" type="string" indexed="true" stored="true" required="false" />
<field name="city" type="string" indexed="true" stored="true" required="false"/> 
<field name="owningComm" type="integer" indexed="true" stored="true" required="false" multiValued="true" />

The combination of type and id determine which resource (either page or file download) has been requested.

Web user interface for DSpace statistics

...

Property:

solr.log.server

Example Value:

solr.log.server = http://127.0.0.1/solr/statistics

Informational Note:

Is used by the SolrLogger Client class to connect to the Solr server over http and perform updates and queries. In most cases, this can (and should) be set to localhost (or 127.0.0.1).

To determine the correct path, you can use a tool like wget to see where Solr is responding on your server. For example, you'd want to send a query to Solr like the following:

Code Block
wget http://127.0.0.1/solr/statistics/select?q=*:*

Assuming you get an HTTP 200 OK response, then you should set solr.log.server to the '/statistics' URL of 'http://127.0.0.1/solr/statistics' (essentially removing the "/select?q=:" query off the end of the responding URL.)

Property:

solr.spiderips.urls

Example Value:

solr.spiderips.urls =

Code Block
http://iplists.com/google.txt, \
http://iplists.com/inktomi.txt, \
http://iplists.com/lycos.txt, \
http://iplists.com/infoseek.txt, \
http://iplists.com/altavista.txt, \
http://iplists.com/excite.txt, \
http://iplists.com/misc.txt, \
http://iplists.com/non_engines.txt


<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="89310670c299d684-73ba0705-42ae4532-a42fba70-4d0044c01619db4e78d76bcd"><ac:plain-text-body><![CDATA[

Informational Note:

List of URLs to download spiders files into [dspace]/config/spiders. These files contain lists of known spider IPs and are utilized by the SolrLogger to flag usage events with an "isBot" field, or ignore them entirely.
]]></ac:plain-text-body></ac:structured-macro>
The "stats-util" command can be used to force an update of spider files, regenerate "isBot" fields on indexed events, and delete spiders from the index. For usage, run:


Code Block
dspace stats-util -h

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="542b611ef79cfbfc-9a87846d-44b3401a-a845a874-8d366156e0cdf27bbccdb308"><ac:plain-text-body><![CDATA[from your [dspace]/bin directory

]]></ac:plain-text-body></ac:structured-macro>

Property:

solr.dbfile

Example Value:

solr.dbfile = ${dspace.dir}/config/GeoLiteCity.dat

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="bd7c2f0f1e01445b-58cfe489-42b24484-9dad8939-35b4d7b8d478a4298a64dd6d"><ac:plain-text-body><![CDATA[

Informational Note:

The following referes to the GeoLiteCity database file utilized by the LocationUtils to calculate the location of client requests based on IP address. During the Ant build process (both fresh_install and update) this file will be downloaded from [http://www.maxmind.com/app/geolitecity] if a new version has been published or it is absent from your [dspace]/config directory.

]]></ac:plain-text-body></ac:structured-macro>

Property:

solr.resolver.timeout

Example Value:

solr.resolver.timeout = 200

Informational Note:

Timeout in milliseconds for DNS resolution of origin hosts/IPs. Setting this value too high may result in solr exhausting your connection pool.

Property:

useProxies

Example Value:

useProxies = true

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="041ae3b705ef1714-456fa493-4ef0467e-a5dab2a8-18a30909ccd4755c421bf113"><ac:plain-text-body><![CDATA[

Informational Note:

Will cause Statistics lohging to look for X-Forward URI to detect clients IP that have accessed it through a Proxy service.  Allows detection of client IP when accessing DSpace. [Note: This setting is found in the DSpace Logging sesction of dspace.cfg]

]]></ac:plain-text-body></ac:structured-macro>

Property:

statistics.item.authorization.admin

Example Value:

statistics.item.authorization.admin = true

Informational Note:

When set to true, only general administrators, collection and community administrators are able to access the statistics from the web user interface. As a result, the links to access statistics are hidden for non logged-in admin users. Setting this property to "false" will display the links to access statistics to anyone, making them publicly available.

Property:

solr.statistics.logBots

Example Value:

solr.statistics.logBots = true

Informational Note:

When this property is set to false, and IP is detected as a spider, the event is not logged.
When this property is set to true, the event will be logged with the "isBot" field set to true.
(see solr.statistics.query.filter.* for query filter options)

Property:

solr.statistics.query.filter.spiderIp

Example Value:

solr.statistics.query.filter.spiderIp = false

Informational Note:

If true, statistics queries will filter out spider IPs -- use with caution, as this often results in extremely long query strings.

Property:

solr.statistics.query.filter.isBot

Example Value:

solr.statistics.query.filter.isBot = true

Informational Note:

If true, statistics queries will filter out events flagged with the "isBot" field. This is the recommended method of filtering spiders from statistics.

...