On this page:[Show/Hide]
- 1 MindLink Desktop Troubleshooting
- 1.1 Login Errors
- 1.2 Connection Errors
- 1.3 Other Errors
- 1.3.1 A Service fails to start
- 1.3.2 The MindLink Desktop Login Screen fails to load, you get a 401 error, or a Connection Refused error
- 1.3.3 The Connector Service fails to start with a 407 Proxy Authentication Required error
- 1.3.4 Connector service fails to start with the following error
- 1.3.5 On logging in, the loading screen hangs
- 1.3.6 An “Unspecified Error” occurs during login
- 1.3.7 An “Unable to contact Messaging Server” error occurs during login
- 1.3.8 An “Unable to contact Connector Service” error code occurs during login
- 1.4 Voice Calling
- 1.4.1 Diagnosing common local issues
- 22.214.171.124 Microphone/recording device access
- 126.96.36.199 Cannot select recording/reproduction device
- 188.8.131.52 Internet Explorer cannot perform calls
- 184.108.40.206 Windows OS edition N, KN and similar
- 220.127.116.11 Firefox version 63
- 18.104.22.168 Edge browser does not use TURN server
- 22.214.171.124 Internet Explorer only uses SfB Edge server for the first call
- 126.96.36.199 Multiparty voice conferencing support
- 188.8.131.52 Video call support
- 184.108.40.206 Putting calls on hold
- 1.4.2 Looking for the MindLink logs
- 1.4.3 Diagnosing network issues
- 1.4.4 Client compatibility
- 1.4.1 Diagnosing common local issues
- 2 MindLink Mobile Troubleshooting
- 2.1 Definitions
- 2.2 General Points of Failure
- 2.3 Determining Failure Origin
- 2.4 Device Logs
- 3 MindLink WebPart Troubleshooting
- 4 System Monitoring
1 MindLink Desktop Troubleshooting
Error messages will be displayed if the application encounters any problems. The error message can be used to determine what the problem is. See Figure 114 for an example error dialog.
Figure 114: An error message
1.1 Login Errors
When login is not successful, the application will display error messages to inform the user of the cause of the problem. The following is a list of login problems which will cause an error message to be displayed:
- Invalid username or password
- Password has expired
- Missing username or password
- Missing SIP address
- Unable to contact connector service
- Unable to connect to messaging server
- Encountered a server error
- Received invalid data from the server
- An unrecognized error
1.2 Connection Errors
A connection problem will occur when the application is disconnected from the server, the session has expired, or has an associated error. When this happens, the application will attempt to reconnect several times (with increasing delays). If reconnection is successful, the application will continue to operate as usual. If reconnection is unsuccessful, an error will be displayed, which will prompt the user to login again.
When reconnection can be established without re-login, the presence of the user will not change and they will appear online all the time. Messages will be retrieved when the user reconnects. However, if the reconnection fails and the user is required to re-login, they will appear offline and messages sent or received during that time will be lost without any explicit notification to the sender of the message.
1.3 Other Errors
1.3.1 A Service fails to start
Check the logs. If there is no logging directory, ensure the service account has write access to the install directory. Ensure the service account has full rights on the database.
Check that your load balancer or proxy is forwarding requests correctly (to the correct ports, etc.). Ensure that the certificates are bound correctly on the Connector service hosts.
1.3.3 The Connector Service fails to start with a 407 Proxy Authentication Required error
This typically happens when the environment is configured to run with a web proxy. The connector service must be configured to use the web proxy. Add the following config to the root element of the
Connector Service’s configuration file (Formicary.Fcf.Services.ConnectorService.exe.config):
1.3.4 Connector service fails to start with the following error
Microsoft.Rtc.Internal.Sip.TLSException: CertificateInfoNative::AcquireCredentialsHandle() failed; HRESULT=-2146893043
Most often, it means that the process does not have permission to access the certificate you are using to authenticate for transport layer security (TLS). This can be resolved by giving the service account read access to the correct certificate key file located at “C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys”.
1.3.5 On logging in, the loading screen hangs
It is likely that the Connector service is failing as it tries to log you in.
Look at the Connector service logs to diagnose the problem. Check that the configured underlying chat system server and ports are correct.
1.3.6 An “Unspecified Error” occurs during login
Use Firebug in Firefox (or a similar diagnostic tool) to confirm that the login AJAX request is successful – i.e. that the response is not a 401 or other HTTP code.
If the response is returning with error code 404, the Connector service is probably not running. Ensure that it is.
If it is returning with another error code, ensure that the certificate is bound correctly on the Connector service machines (use httpcfg or netsh), and that it is configured properly in the Connector service config files. Observe the Connector service logs to confirm that the request is reaching the Connector Service properly.
If it is not returning an error code, it is likely that an error is occurring on the Connector service. Check the Connector service logs to diagnose the fault. If connecting to Microsoft Group Chat Console, SSO is set to true and the error is ‘Tls failure exception’, make sure that in the Connector settings in the Administration Console server port is set to 5061 and transport type to ‘Tls’.
This error also occurs when the type of connector (gcc or ma) is changed and the MindLink Desktop services are not restarted. Ensure that ALL MindLink Desktop services are restarted (only restarting the Connector Service will not do) when the connector type is changed.
1.3.7 An “Unable to contact Messaging Server” error occurs during login
Confirm that the Connector Service is configured to talk to the underlying chat system correctly – check the sever name, port, transport type and sip address.
1.3.8 An “Unable to contact Connector Service” error code occurs during login
It is likely that the Connector service is not running, or a firewall is blocking requests. Use Firebug in Firefox (or similar diagnostic tool) to ascertain what the HTTP response code of the login AJAX request is.
If the error is 404, the Connector service is probably not running, or it is listening on the wrong port or path. If the error is 401 or 500, the certificate may not be installed correctly. Ensure that it is installed on the Connector service machines, that the Connector service machine has the certificate bound to the appropriate port (use httpcfg query ssl), and that the Connector service is configured to use the certificate and HTTPS in its config files.
Observe the Connector service logs to confirm that the request is reaching the Connector service properly.
Enable debug mode by changing the client.debug setting to true in the Administration Console. Use Firebug in Firefox (or similar diagnostic tool) to examine the logs. Report logs to Formicary.
1.4 Voice Calling
This document provides troubleshooting guidance for the voice feature of MindLink web client, first introduced with version 18.7.
It is broken down into three sections: a section on local client setup problems such as access to microphone devices. A section on how to get voice diagnostic error information from the MindLink client and server which can help identify the nature of the problem. A section on troubleshooting networking issues between different client endpoints and supporting servers. Finally, a section on compatibility with other clients (such as the Microsoft SfB desktop client).
Voice calling in MindLink occurs over two separate sessions: a session with the SfB server, and a direct connection between the client endpoints where the audio media is streamed between the clients. Under certain network conditions a direct connection between the client endpoints is not possible and additional server components are required to support these cases, see the voice feature technical overview document for more information.
1.4.1 Diagnosing common local issues
This section details various issues with client setup, as well as known issues and limitations.
220.127.116.11 Microphone/recording device access
The MindLink web client will attempt to use the default audio recording and reproduction device available on the machine it is running on. There are several issues to take into account in these cases:
- Missing audio recording device (microphones): the client will fail to capture user audio and will abort the call. The client should display a warning after logging on if it cannot find a microphone device. In modern browsers, it is necessary to grant audio recording permissions from the browser tab in order for the voice feature to work. Enterprise policies on granting such permissions may prevent the feature from working.
- Occupied audio recording device: a common pitfall is attempting to establish a call between two client endpoints from the same machine (different browser tabs, different browsers etc), microphone access is in often exclusive, so if there is a browser or other application currently capturing microphone input, then attempting to access the device from another application will result in an error or incorrect behavior. This can also occur when two remote desktop sessions are being used for testing, each running from the same physical machine.
It is therefore recommended that basic feature testing and setup is performed using two separate physical machines, each with their own recording and reproduction devices, without making use of remote desktop.
18.104.22.168 Cannot select recording/reproduction device
There is currently no support for manually selecting which reproduction/recording device to use. The MindLink client will use the system default devices.
22.214.171.124 Internet Explorer cannot perform calls
The MindLink client uses the webrtc API in modern browsers to perform calls, voice calling is supported in Internet Explorer 10 and 11 which do not implement this modern API by using Microsoft's "Skype Meetings App" (also known as "Skype for Business Web App Plugin") browser plugin (see the voice feature technical overview document for more information). This plugin must be separately installed on the same machine running Internet Explorer for voice calling to work. A warning should be shown after logging on in case the client cannot detect the plugin. In case the plugin is installed while the client is open, the client must be restarted by refreshing the page or logging out after installing the plugin.
Outdated and unsupported versions of this plugin are distributed with the SfB 2015 server installation but will not work with MindLink web client, the version required by MindLink web client can be found at: https://swx.cdn.skype.com/s4b-plugin/126.96.36.199/SkypeMeetingsApp.msi.
188.8.131.52 Windows OS edition N, KN and similar
Editions N (and related) of Windows require the appropriate media pack for the OS version in order for Firefox and Internet Explorer to be able to capture audio. For more information see: https://support.microsoft.com/en-gb/help/3145500/media-feature-pack-list-for-windows-n-editions.
184.108.40.206 Firefox version 63
Version 63 of Firefox introduced a bug with the webrtc API, this affects MindLink web client when receiving calls from Internet Explorer. This does not occur on Firefox62 and lower, and was fixed for Firefox64 and higher, see here for more details: https://bugzilla.mozilla.org/show_bug.cgi?id=1495569.
220.127.116.11 Edge browser does not use TURN server
This is a known issue with version 18.7 of the MindLink client and will be fixed in version 18.8.
18.104.22.168 Internet Explorer only uses SfB Edge server for the first call
This is a known issue with version 18.7 of the MindLink client and will be fixed in version 18.8.
22.214.171.124 Multiparty voice conferencing support
Multiparty voice will be added in a future release.
126.96.36.199 Video call support
There is currently no support for video calls, only voice calls are supported.
188.8.131.52 Putting calls on hold
There is currently no support for putting calls on hold, or handling being put on hold by the remote peer.
1.4.2 Looking for the MindLink logs
This section explains what to look for in the client and server logs to help diagnose voice issues.
184.108.40.206 Local client error
Local client logs can help diagnose issues with the establishment of the peer to peer connection between client endpoints.
To enable client logging, simply add "?enableLogging=true" to the end of the browser tab URL (e.g. "https://mindlink.mycompany.com/?enableLogging=true"). Once enabled, use the browser developer tools (which can be accessed by pressing the F12 key in most browsers) and navigate to the "console" section of the developer tools.
Simply filtering the console messages via the developer tools (for Chrome and Firefox CTRL + F or cmd + F) looking for "voice" or "audio" should show the relevant log messages. Error messages will contain an error stack showing the original API error. Depending on the browser, this error will be generated by the webrtc API or by the Skype Internet Explorer plugin (see the voice technical overview document for more information), since the first is implemented in slightly different ways in every browser, errors will vary, but can contain useful information.
The Edge and Internet Explorer browsers are very slow at displaying logs, and due to the volume of messages logged, it can take a couple of minutes after starting a call before the log messages and errors relating to the call appear in the logs. Furthermore, the Internet Explorer browser offers no logs filtering feature, however in Internet Explorer 11 it is possible to right click the console log and select "Copy all" to copy all log messages to a text editor or log monitor. Additionally, Edge and Internet explorer both tend to truncate long log messages, this means long SDP documents may be truncated and not all candidates will be visible in the client. These should however be fully logged by the server too. It is possible to cross-reference the call ID from the client logs with the server logs to find the full SDP documents for a given call.
220.127.116.11 Remote client error
If the error that prevented the call from establishing is generated on the peer client, an error will appear at the top of the client explaining the issue. Similarly, if the call is attempted and accepted on the peer end, but after a while fails then the client logs will contain something similar to the following: "Handling removal of current call 'some ID' by transitioning to ended". This line is an indication that the error occurred on the other client, and its logs should be inspected for errors or other messages. This could also indicate a failure of the peers to reach each other, indicating the need for a STUN/TURN server.
In certain situations the error will be not between the peer connection, but with the connection to the SfB server. In these cases the server logs may contain useful information.
The MindLink service logs are normally kept inside the MindLink server installation directory. It may be useful to ensure that the service is configured to log at "Verbose" (maximum) level. This can be changed from the management tool and requires a service restart.
Like with the client logs, filtering the log file for lines containing "voice" or "audio" should show the relevant log messages, again, errors will be logged with an error stack.
1.4.3 Diagnosing network issues
Since the connection over which the audio is streamed between client endpoints is direct, there are several issues that can occur connecting two clients due to network architecture. This section explains how to troubleshoot peer connection and STUN/TURN server connections (see the voice technical overview for more information).
This section is divided between diagnosing direct or STUN-ip client connection and troubleshooting STUN/TURN server connection. The process essentially involves finding the IP address of the remote machine and filtering the network packets capture by this IP address.
The IP addresses will either be found in the call SDP documents (logged by the client and server), or will be the IP of the Edge or STUN/TURN server. Additionally, it is possible to filter for specific UDP or TCP ports which can help narrow down the range of filtered logs further, protocol, IP and port are all described in the SDP.
18.104.22.168 Peer connection troubleshooting using WireShark
Wireshark is a free tool to analyze network traffic to/from the local machine. It is a very powerful and versatile tool.
In this section we will show examples of how to use Wireshark to diagnose network issues. The basic process to use in every case involves a few steps:
- Identify the network device on the machine that will be actually used to attempt the connection, select this device as the Wireshark live capture target.
In general, this is the network interface on the local machine that is used to connect to the network, a WiFi device or ethernet controller.
- Find the IP addresses of the remote call peer (the machine of the endpoint which is being called, or has started the call). These can be found from the MindLink client logs from the remote SDP document.
- Filter Wireshark live capture with these IP addresses (assuming there is no other network traffic between the local machine and the IP this should show only the packets relevant to performing the call connection).
The screenshot shows a "sunny day" scenario where two local IP addresses establish a connection directly as they are both directly reachable.
If no traffic at all appears between the machine and the target IP address this means neither of the two is actually attempting the connection. Alternatively, requests sent from the local machine appear but no response can be seen from the peer's IP addresses. This can point to a firewall (on either host machine, or the NAT router) blocking the connection or similar.
If traffic appears but if only TCP RST (connection rejected) messages or ICMP "destination unreachable" messages are shown this means the IP address likely belongs to a different private network from the local machine, in such cases a STUN/TURN server is required in order for the machine to reach the peer.
22.214.171.124 Standard STUN/TURN server setup troubleshooting
Analyzing STUN/TURN server setup via Wireshark involves more or less the same exact steps as analyzing direct peer connection, except the IP address of the server will be used to filter the capture log.
The previous problematic cases apply here as well.
For each STUN/TURN server configuration added from the server management tool, the clients will attempt to establish a connection to the server every time a call is started or answered. This can lead to very long wait times as each server is contacted one after the other. It is recommended that no more than two servers are configured to keep call initialization time minimal.
126.96.36.199 SfB Edge server setup troubleshooting
This section details how to troubleshoot Internet Explorer plugin to use the SfB Edge server. Running MindLink on Internet Explorer will attempt to use the SfB Edge server automatically. It will also attempt to use any STUN/TURN servers configured in the management too, but will fail to utilize them properly in case they do not support the MS-TURN protocol.
A token identifying the user endpoint with the SfB Edge server is retrieved from the MindLink server via the SfB frontend, this is then forwarded to the client and normally contains two separate addresses for the Edge server, one reachable from the internet and another reachable from the intranet. The MindLink web client attempts to use both and uses the first one that responds.
The process of client authentication against the SfB Edge server follows Microsoft's own extension of the STUN protocol spec MS-TURN. This extension is incompatible with the standard protocol implemented by STUN/TURN servers and clients (such as webrtc clients implemented by Chrome and Firefox and TURN server implementations like coturn).
This means the SfB Edge server cannot be used by MindLink web client (webrtc) on modern browsers, but can be used by the Internet Explorer plugin as this was developed by Microsoft primarily for compatibility with SfB. For this reason, the plugin can only use the Edge server and cannot use standard STUN/TURN servers as it only supports the extended protocol. The MindLink web client cannot use the Edge server to receive the SfB client media because the SfB Edge server requires authentication via MS-TURN, however the SfB client can use a standard STUN/TURN server to receive the browser media when the TURN server does not require authentication from the receiving party (coturn by default, for example, only requires the sending party to authenticate).
Commercial STUN/TURN server implementations that support both MS-TURN as well as standard STUN/TURN and shim between can help alleviate these issues.
Again, the cases mentioned before apply to the SfB Edge server. In addition, here are shown cases displaying what happens when a webrtc client attempts to use the Edge server, and what happens when the Internet Explorer plugin attempts to use a standard TURN server instead of the Edge server.
The following shows that attempting to use a TURN server that does not support the MS-TURN extensions from the IE plugin results in rejection responses.
1.4.4 Client compatibility
This section details cross client compatibility between MindLink web client and other clients leveraging an on premise SfB deployment.
188.8.131.52 SfB Windows desktop client
The MindLink web client is supported against the SfB 2016 desktop client versions 16.0.4639.1000 and higher. The client version can be found under "Help" -> "About Skype for Business" on the client itself (should also be visible under the installed programs view on Windows).
Older clients, such as the Lync 2013 desktop client, will only work with MindLink web client when this runs on Internet Explorer. The reason is that the only security mechanism supported by older SfB/Lync client versions is not supported by the webrtc API in modern browsers (see the voice technical overview for more details).
Attempting to use an old Sfb/Lync client will result in the webrtc client rejecting the SDP due to this missing a fingerprint line or (equivalently) DTLS security details. This is because originally the Lync client had no support for this security mechanism, but this was added in more recent versions. DTLS is the only mechanism modern browsers are meant to support.
Additionally, it is possible outdated Lync clients will reject the SDP originated from the webrtc client with no explanation. This will likely show as a SIP 488 "Not acceptable here" error in the MindLink server logs.
184.108.40.206 SfB macOS desktop client
The macOS version of the Sfb/Lync client is currently unsupported. Support may be investigated in the future provided interest for it exists.
220.127.116.11 SfB mobile clients
The SfB Android and iOS mobile apps are only supported for inbound calls on modern browsers, this is due to the MS-TURN extensions to the STUN protocol (see above): the mobile client does not strictly follow MS-STUN when receiving a call, but strictly follow the protocol (which the native browser API does not support) when making a call outbound.
This results in the SfB mobile client not responding to packets received from the web client on modern browsers, and can be observed via WireShark.
18.104.22.168 MindLink Mobile clients support
There is currently no voice support for the MindLink mobile clients.
2 MindLink Mobile Troubleshooting
Apple Push Notification Service – responsible for push notifications for iOS clients
BlackBerry Mobile Data Service – The BlackBerry service that interoperates between the BlackBerry devices and the BlackBerry Enterprise Server.
BlackBerry Administration Service – The BlackBerry service that provides administrative function, including the administration interface.
BlackBerry Enterprise Server – The collective term for the BlackBerry Enterprise infrastructure (MDS, BAS etc)
Central Push Server
A special MDS server that acts as a master, delegating to several slave MDS servers. The Central Push Server should be solely used by MindLink Mobile to communicate with the BES.
An IT Policy is a collection of settings that are defined by an administrator in the BAS and are assigned to devices. Custom IT Policy rules are used by MindLink Mobile to define server connections and global MindLink Mobile settings.
The term used to identify BlackBerry MDS operating logs. The MDAT logs include all device-server communication logs.
The logs can be found by default on the MDS server at:
The term used to identify BlackBerry MDS policy logs. The POLC logs include only IT Policy logs i.e. deployment of applications/policy updates.
The logs can be found by default on the BAS server at:
This is a file installed with every MDS instance that specifies some MDS instance-specific settings. You can find it on the MDS server at:
Formicary Chat Foundation – The collective term for the Route, Connector and Configuration and Monitoring service.
2.2 General Points of Failure
MindLink Mobile has various potential points of failure as shown in Figure 3a.
Figure 115- MindLink Mobile Failure
These points of failure can be categorized as follows:
1. Device – Affects communication at ‘a’ and ‘b’.
2. MDS – Affects communication at ‘a’, ‘b’, ‘c’ and ‘d’.
3. Connector Service (Mobile Broker) – Affects communication at ‘c’ and ‘d’.
2.3 Determining Failure Origin
It is best to identify a point of failure one-at-a-time, starting with failures closest to the device first i.e. if the mobile broker and the MDS are failing, handle the MDS failure first.
In order to identify a point of failure successfully you should start with the Mobile Broker logs and follow the process in Figure 115.
Essentially you will investigate the causes of failure by looking at the following log files:
1. The Mobile Broke Logs : %Program Files%\MindLink Software\ConnectorService\Logs\Connector.log
2. The MDS MDAT log files : %Program Files%\Research in Motion\BlackBerry Enterprise Serevr\Logs\\_MDAT_.txt
3. The MDS POLC logs : %Program Files%\REasearch in Motion\BlackBerry Enterprise Serevr\Logs\\_POLC_.txt
4. The Device Logs : accessed using the shortcut alt+l, alth+g, alt+l, alt+g on the device
When debugging any issue, it is best to start with the Mobile Broker logs. This will tell you whether the issue is BES/device-side or FCF-side.
All issues are almost always configuration problems. If you identify that a problem is FCF-side, consult the FCF/MindLink Desktop troubleshooting guide, otherwise this guide should contain all known problems and their solutions.
Once you can get a device logging in, you have confirmed that the device, MDS and MindLink Mobile Broker are able to communicate. Any further issues will be general MDS/Broker/FCF problems that will need to be individually investigated using the available logs.
Always look at the “Common Issues” section before using the decision flow diagrams as it is likely to lead to a faster resolution.
2.3.1 Problem to Solution Decision Flow of Connector Service (MindLink Mobile Broker)
Figure 116- Broker log decision flow
Points of failure specific to the BlackBerry
Figure 117- MDS log decision flow
2.3.2 Connector Service Common Issues
No device pin in requests
This has the following causes:
1. The MDS is not set to forward device pin in http headers. Fix this by setting the application.handler.http.header key value in the rimpublic.property file to email,pin.
2. The MDS is set to forward device pin, but the header domain regex does not match the Broker host. Fix this by setting the application.handler.http.header.domain key value in the rimpublic.property file to a regex that will successfully match the Broker host (this is case sensitive and must match the host name that appears in the MindLink Mobile formicary.MindLink Mobile.server_address IT Policy rule).
Device requests are to the correct host but a ridiculous port
This has the following causes:
1. The MindLink Mobile formicary.MindLink Mobile.server_port IT Policy rule value is set to an incorrect port. Adjust the port, wait for the change to be pushed to the device, and retry.
2. The formicary.MindLink Mobile.server_port IT Policy rule type is not set to integer. Usually it defaults to string, and this then causes the device to interpret the port as the sum of the ASCII character codes. Change the type to integer and retry.
400 – Bad Request
This may be an issue with the specification of push reliable ports on the BAS. Ensure that the device port the server is trying to reach (the MindLink Mobile Target Port configuration value (see Section 3.8)) is set as a reliable push port in the BAS:
The mobile client does not receive notifications from the server
This might have the following causes:
1. The IT Policy incorrectly specifies the server connection details. This prevents communication between the BES and the MindLink Mobile host server, therefore the Blackberry clients cannot communicate with the MindLink Mobile service.
Follow the steps detailed in section 4.5 Blackberry Enterprise Server Configuration.
Ensure that the details provided to the BES match those specified in the server-side configuration files.
2. The Blackberry Enterprise Server date and time does not match the MindLink Mobile host server date and time. This causes notifications to be immediately discarded by the BES.
Ensure that the date and time settings for the BES and the MindLink Mobile host server match.
2.3.3 MDS Deployment Common Issues
Device reported general error.
Try deploying the same version of the software using JavaLoader preferably to the failing device or an identical OS version.
The failure should be reported in more detail.
If the failure is an invalid Java header, this means that a newer version of the rapc compiler and rim_api.jar files were used when building. A new build is required using an earlier version of the rapc compiler matching the device OS version.
Note: We support OS 4.6 and above.
Controlled Access Exceptions
Caused by unsigned cod files. This requires rebuilding, resigning, and redeployment.
Initialisation fails showing “invalid URL or security settings”
Usually caused by specifying an invalid URL for the “server_address” IT Policy setting. It must be a full URL of the identification service and the protocol must be lowercase e.g.
2.3.4 General Communication Common Issues
This could be one of the following reasons:
1. The Mobile Broker host time is out of sync with the MDS host time. This will cause the push-expiry time set by the Broker to be triggered too soon. Correct this by synchronizing the clocks and extending the Push Notification Timeout MindLink Mobile configuration value (see Section 3.8).
2. The device is out of coverage and the failure was legitimate.
3. The communication between the device and the MDS is suffering from high latency. If this happens a lot and the device is in coverage, it indicates that the network is slow. You can reduce the occurrence of the issue by extending the Push Notification Timeout MindLink Mobile configuration value (see Section 3.8).
Usually this means that the device closed the push listener, i.e. the application was closed.
Consult the device logs for more details.
This is an issue with the MDS itself and if it is a major issue for the client, it should be escalated to RIM. Ensure that they are running the latest update to the BES version they are using.
This is an issue with the specification of push reliable ports on the BAS. Ensure that the device port the server is trying to reach (the MindLink Mobile Target Port configuration value (see Section 3.8)) is set as a reliable push port in the BAS:
Restart the MDS after making the change.
2.4 Device Logs
2.4.1 Blackberry 5 Devices
When running MindLink Mobile on a Blackberry 5 device the logging entries are entered into the devices native log file – this can be accessed by holding down the “alt” key and alternatively pressing L & G twice (alt + L , G , L , G).
In the device logs you should pay particular attention to bold entries of the form:
E MindLink Mobile – xxxxxmessage
S MindLink Mobile – xxxxxmessage
These may also include stack-traces which you can access by selecting the entry.
2.4.2 Blackberry 10 Devices
Blackberry 10 devices work slightly differently in regards to logging – with Blackberry 10 devices the MindLink Mobile app creates its own log file – this can be found in Documents/Mindlink-log.txt
2.4.3 iOS Devices
By going to ‘application settings’ on your iPhone, you can ‘enable logging’. By enabling logging, it is possible to receive an internal log to your device console. The internal log can be viewed by downloading a console app from the app store.
If the application crashes, the logs for this are transmitted to Formicary’s TestFlight account. For more information on TestFlight, see: https://testflightapp.com/.
2.4.4 Android Devices
For Android users, the logs are located on the SD card of the device in MindLink\log.txt (if an SD card is not present then the logs will log to the device memory – sdcard\MindLink\log.txt)Performance Counter Logs
3 MindLink WebPart Troubleshooting
The WebPart writes diagnostic messages to the SharePoint ULS log, which generally writes to files in: %ProgramFiles%\Common Files\Microsoft Shared\Web Server Extensions\14\LOGS
If you experience any issues with the application, in the first instance contact your local IT support team. If you have subscribed to Support and Maintenance and require additional assistance from the Formicary Collaboration, please email the Support Team at firstname.lastname@example.org stating the nature of the problem and including your contact details.
4 System Monitoring
4.1 Performance Counters
The MindLink Server exposes a custom suite of Windows performance counters that can be used to monitor the usage and health of the system.
If Performance Counter Logs are not working, ensure that the service account is a member of the local Performance Logs Users group on the server.
the performance counter suite can be used to indicate general usage statistics as well as to detect outages or other problems, via the number and rate of errors thrown.
In addition, the MindLink server is a .NET service and hence exposes the standard suite of .NET performance counters. It is recommended to monitor at least the standard Windows memory usage and CPU performance counters, as well as the MindLink performance counters that report error rates, as a general indication of service health.
The Connector service exposes several performance counters. The new category in which they are found is called “CollaborationFoundation”. The performance counters and how they are updated are as follows:
Desktop & MobileChat:
- ActiveSessions: The active user sessions connected into webchat/mobilechat.
- Active Subscriptions: The number of current presence subscriptions.
- Total Subscriptions: - The total number of subscriptions made so far.
- Total User Information Requests Calls: - The number of user information lookups made so far.
There are global performance counters for authentication as well as performance counters specific to each authentication mechanism (Password, Windows and HTTP Header Authentication). Each global performance counter listed below has a similar counter specific to the authentication mechanism in use.
- Authentication Requests: The number of authentication requests made to the server.
- Authentication Requests per Second: The number of authentication requests made to the server per second.
- Failed Authentication Requests: The number of authentication request made to the server which have failed.
- Failed Authentication Requests Per Second: The number of authentication requests made to the server per second which have failed.
- Last Authentication Time Taken Milliseconds: The number of milliseconds the last made authentication request to processed by the server.
- Average Authentication Request Time Seconds: The average number of seconds to process authentication requests by the server.
-Active Persistent Connections: An up-to date number of active Tcp Connections.
-Mobile Session Event Filter Percentage: The percentage of events received in the FCF service that are sent to the client.
-Push Data Sent: The cumulative total number of bytes sent via the push connection.
-Push Messages Sent: The cumulative total number of messages sent via the push connection.
-Socket Data Received: The cumulative total number of bytes received via the socket connection.
-Socket Messages Received: The cumulative total number of messages received via the socket connection.
-Socket Messages Sent: The cumulative total number of messages sent via the socket connection.
-GroupMessagesReceivedCounterKey : Group messages received.
-GroupMessagesReceivedPerSecondCounterKey: Group messages received per second.
-GroupMessagesSentCounterKey: Group messages sent.
-GroupMessagesSentPerSecondCounterKey: Group messages sent per second.
-RemoteGroupJoinsCounterKey: Remote group joins.
-RemoteGroupJoinsPerSecondCounterKey: Remote group joins per second.
-RemoteGroupLeavesCounterKey: Remote group leaves.
-RemoteGroupLeavesPerSecondCounterKey: Remote group leaves per second.
-GroupMetaDataUpdatesCounterKey: Group meta data updates.
-GroupMetaDataUpdatesPerSecondCounterKey: Group meta data updates per second.
-GroupIdLookupsCounterKey: Group ID lookups counter.
-GroupIdLookupsPerSecondCounterKey: Group id lookups per second.
-JoinedGroupsCounterKey: Joined groups counter.
-LocalGroupJoinsCounterKey: Local group joins counter.
-LocalGroupJoinsPerSecondCounterKey: Local group joins per second.
-LeftGroupsCounterKey: The left groups counter.
-LocalGroupLeavesCounterKey: The local group leaves.
-LocalGroupLeavesPerSecondCounterKey: Local group leaves per second.
-InstantMessagingCallsCounterKey: Instant messaging calls counter.
-InstantMessagesSentCounterKey: Instant messages sent counter.
-InstantMessagesSentPerSecondCounterKey: Instant messages sent per second.
-InstantMessagesReceivedCounterKey: Instant messages received counter.
-InstantMessagesReceivedPerSecondCounterKey: Instant messages received per second counter.
-LocalComposingStateChangesCounterKey: Local composing state changes counter.
-LocalComposingStateChangesPerSecondCounterKey: Local composing state changes per second counter.
-RemoteComposingStateChangesCounterKey: Remote composing state changes.
-RemoteComposingStateChangesPerSecondCounterKey: Remote composing state changes per second.
-FileUploadsCounterKey: The file uploads counter.
-FileUploadsPerSecondCounterKey: The file uploads counter.
-BytesUploadedCounterKey: The bytes uploaded counter.
-BytesUploadedPerSecondCounterKey: The bytes uploaded counter key.
-FileDownloadsCounterKey: The file downloads counter.
-FileDownloadsPerSecondCounterKey: The file downloads counter.
-BytesDownloadedCounterKey: The local group leaves per second.
-BytesDownloadedPerSecondCounterKey: The local group leaves per second counter.
-UserSearchesCounterKey: The user searches counter.
-UserSearchesPerSecondCounterKey: The user searches counter.
-GroupSearchesCounterKey: The group searches counter.
-GroupSearchesPerSecondCounterKey: The group searches per second counter.
-HistorySearchesCounterKey: The history searches counter.
-HistorySearchesPerSecondCounterKey: The history searches counter.
-PreferencesSavesCounterKey: The preferences saves counter.
-PreferencesSavesPerSecondCounterKey: The preferences saves per second counter.
-ExchangeWebServicesCallsCounterKey: The exchange web services calls counter.
-ExchangeWebServicesCallsPerSecondCounterKey: The exchange web services calls counter.
See Figure 118 for the screen shot of the performance counters.
Figure 118: Performance Counters
Tools such as Microsoft’s Performance Monitor (perfmon.exe) can be used to view the value of the counter and generate reports.
Please note to view Performance Counters from a remote computer, the Performance Logs and Alerts firewall exception must be enabled on the remote computer.
4.2 MindLink Logging
The MindLink Server writes log messages to a text-based log file. The location of this file is configurable via the Management Center.
Each log is written at one of the log levels:
- Verbose – Detailed information about each action being processed, for diagnostics and troubleshooting.
- Info – Post-mortem information about significant completed operations, for usage reporting.
- Warning – Notifications of environmental, configuration, or operation abnormalities that may indicate problems – immediate, or in the future – for health monitoring.
- Error – Information about erroneous application behaviour for troubleshooting and health monitoring.
The minimum log level – as ordered above – of logs that will be written to the log file is configurable in the Management Center. All logs below this level will be discarded.
Logging at Info is recommended for production.
4.3 Skype For Business Logging
The MindLink Server uses the Microsoft UCMA SDK and Persistent Chat SDKs to integrate with Skype for Business. Each of these SDKs emit Event Tracing for Windows (ETW) logs that can be captured by the SFB Centralized Logging Service or Debugging Tools (OCS Logger) etc. The SfB core component or debugging tools must be installed on the MindLink host server, respectively.
For Lync 2010 and OCS 2007 R2 integration, MindLink uses the Group Chat SDK. This does not emit ETW logs but instead can be enabled to write to a text-based log file, distinct from the standard MindLink log file. The Management Center can be used to configure whether logging from the SDK is enabled and to what location.
These logs can be used to diagnose and troubleshoot integration problems between SfB and MindLink.
4.4 Service Status
The MindLink Server runs as a Windows Service. Any environmental or configuration problems encountered during startup will cause the service to abort startup and stop immediately. Once started, further errors – environmental or otherwise – will not cause the service to stop.
The MindLink Server exposes an HTTP health-check Info Service that can be used by external monitoring systems or load balancers to verify that the service is started, healthy and operation.
By default, this service is available at http://:9081/infoservice/status. Making a GET request to that request will return an HTTP status code of 200.
When the MindLink Mobile Server is deployed in a pool, each node must be configured to connect to a Microsoft SQL Server database layer. When a node loses connectivity to the database for any reason it will declare itself unavailable and remove itself from the pool, until database connectivity is restored. While unavailable, the node will reject new logon requests and hence should be disregarded as a candidate node by the load balancer.
The Info Service will return HTTP status code 503 while the node is disconnected from the database. This status code can be used by the load balancer to maintain the set of candidate pool members, and by any external monitoring systems to detect database connectivity failure.
4.5 Usage Monitoring
The MindLink performance counter suite indicates general usage of the system including number of messages sent and received, joined groups etc.
Mobile, Desktop, SharePoint
The ActiveSessions counter records the number of user sessions connected to the node. In Skype for Business terms, this is the number of SfB endpoints maintained by the server on behalf of logged on users.
For Mobile, this number represents the number of users who have a logged on server-side session, not necessarily users who are actively using the app in the foreground on their device. For Desktop and SharePoint, this is the number of active browser sessions open at the current time.
4.6 Access Restriction
Mobile, Desktop, SharePoint
If a MindLink license has been purchased for a subset of the enabled Skype for Business users, it is possible to restrict access such that only the licensed users are able to log on.
- Desktop and SharePoint – it is necessary to restrict access as the license mechanism will periodically count the users who could log on to MindLink (“enabled for MindLink”) and compare this against the licensed number of users.
If the number of users enabled for MindLink is greater than the licensed number, the MindLink Server will begin deny users logging on until the number of users enabled for MindLink is reduced.
- Mobile – The number of users with a logged-on mobile session is dynamically compared to the number of licensed users. When there are as many logged-on sessions as licensed users, further log ons will be denied until one of the existing sessions is ended.
It may be necessary therefore to ensure that the number of active sessions never exceeds the licensed capacity, by restricting access to the set of users who are licensed to start a session.
During log on, the MindLink Server queries Active Directory to map the credentials provided by the user to their SfB SIP address. This entails making an LDAP search for the SIP-enabled A/D object, given the Security Identifier of the logging on account.
The Management Center offers two ways in which this query can be altered to only succeed for users “enabled” for MindLink.
- Specifying an Active Directory Group
Only SIP-enabled objects that are a member of this group will be allowed to log on. The group may be a Distribution or Security group, with scope such that membership is replicated to the directory server that MindLink will connect to.
- Specifying a custom LDAP Search Filter
Only SIP-enabled objects that match the given search filter will be allowed to log on. The default search filter may be augmented to only match users with a custom “MindLinkEnabled” attribute, for example.
In either case, users whose SIP-enabled objects do not meet the criteria will not be allowed to log on, and they will not be ignored during reconciliation of enabled/active users vs licensed capacity.
Skype for Business
MindLink establishes a standard SfB endpoint for each user session and agent. All data – e.g. messages, presence changes etc – is routed into SfB in the same way as with a native Microsoft SfB endpoint.
As such, standard SfB monitoring, reporting and archiving tools can be used to record user actions on a MindLink endpoint. MindLink endpoints are established with a “MindLinkServer” SIP user agent string.
4.7 User Administration (Desktop, SharePoint, Mobile)
A MindLink session will be interrupted if any of the following administration changes is made to the user account:
- The SIP address of the user is changed.
- The user is moved to a different pool.
- The user is removed from SfB.
The user will be notified that their session has ended and they will be required to re-logon