Storage

SPBM Performance Whitepaper

VMware Storage Policy Based Management (SPBM) is a storage policy framework that helps administrators match VM workload requirements against storage capabilities. SPBM runs as an independent service in the vCenter Server. We recently released a white paper [click here to download] that covers SPBM performance in two sections.

  1. Performance comparison of vCenter 6.5 v/s vCenter 6.0U3

    In the first section, we have covered performance improvements of SPBM in vCenter 6.5 over vCenter 6.0 U3. Here is a chart from the whitepaper showing improvement in one of the SPBM workflows.

  2. Scaling API Latency for vCenter 6.5

    In the next section, we have covered scaling trends of SPBM APIs in vCenter 6.5. To determine scaling patterns we passedincreasing number of entities to these APIs as input . The charts are plotted for number of entities against the time it took to complete the task. Below is one such chart from the whitepaper.

 

The post SPBM Performance Whitepaper appeared first on VMware VROOM! Blog.

Read more..

Big Data Technical Breakout Sessions and Related Events at VMworld 2017

There are manyinteresting technical breakout sessions and other discussions on virtualizing big data being given at the VMworld events this year. To give you a time-ordered roadmap to all of these sessions, here is the list we have built along with their times and dates of delivery. Check in at the event itself for the rooms in which these will be held We will look forward to seeing you there!

VMworld USA- Mandalay Bay-Las Vegas

Sunday, 27th August 2017

[VIRT1971QU] Virtualizing Big Data: Quick Talk: 2:00-2:30pm

This short session gives an overview of the other technical sessions that will be held in the main conference on the following days.

Monday, 28th August 2017

[SER3052BU] How VMware vSphere and NVIDIA GPUs Accelerate Your Organization: 1:00-2:00pm

Big data analytics and machine learning are shown here as two workloads that can exploit the power of GPUs in the datacenter as they emerge into popular use.

[MMC3164BU] How Data Science is Transforming Operations: Introduction to Wavefront byVMware: 2:00-3:00pm

Companies are re-thinking how they monitor and manage their systems, The data science techniques that WaveFront customers like Box, Lyft, Workday and Groupon use will be explored in this talk.

[VIRT1400BU] Real-World Customer Architecture for Big Data on VMware: 2:30-3:30pm

A VMware customer speaks about the technical details of the company’s deployment of Hadoop on vSphere

[FUT2634PU] Big Data for the 99% (of Enterprises): 3:30–4:30pm

A team from VMware’s Office of the CTO shows their work in providing a “spreadsheet for big data” that gives convenience at scale, contrasting with the infrastructures that many companies look to first

[STO3192BU] Deploying Big Data on HCI Powered by VSAN : 4:00-5:00pm

Deployment scenarios, best practices and results from various tests using VSAN as the basis for big data deployments are described in this very interesting talk.

[VIRT1351BU] New Architectures for Virtualizing Spark and Big Data Workloads : 5:30-6:30pm

This talk looks into the viability of using different file systems to the Hadoop Distributed File System (HDFS) and explores a set ofalternative storage mechanisms for that data. These are becoming more common as big data matures on virtualization technology.

Tuesday, 29th August 2017

Lenovo Theater Presentation: 12:00 -12:15pm – at the Lenovo booth on Exhibitor Floor

[FUT2020BU] Wringing Maximum Performance from vSphere for Extremely Demanding Workloads and Customers: 12:30-1:30pm

In this Office of the CTO (VMware) session,the speakerswill share details of worktheyhave done in three extremely performance-critical situations.They will talk about tuning and configuring VMWare ESXi to run a very latency-sensitive financial application, for example.

[VIRT1997BU] Machine Learning and Deep Learning on VMware vSphere: GPUs are Invading the Software-Defined Data Center: 5:30 -6:30pm

As Graphical Processing Units spread further inside the data center, this talk examines their use for virtualizing workloads other than VDI, such as for big data and analytics - and shows how best to make your decisions about using the technologies together.

[VIRT2274GU] Group Discussion on Virtualizing Big Data and Machine Learning : 5:30-6:30pm

This session is a chance for you to bring up your hot topics, opinions and questions about big data and machine learning in a virtualization context.

Wednesday, 30th August 2017

[LDT2800PU] Harnessing the Power of Data in a Virtual World: 9:30 – 10:30am

A groupof VMware’s own experts in processing data to get business value from it will speak about their experiences and lessons learned in doing so here.

[MTE4789VIRT] Meet the Experts Session : 11:15am-12:00pm – Table 7

This is an opportunity to meet with VMware’s big data people in a small group context. Booking your time-slot ahead of the meeting is advised here.

Thursday, 31st August 2017

[VIRT1445BU] Extreme Performance Series: Fast Virtualized Hadoop and Spark on All-Flash Disks : 10:30-11:30am

Using vSphere 6.5 and the latest Intel hardware, VMware’s performance engineering lead on big data shows further evidence of the high performance and flexibility you can achieve by virtualizing your big data deployments. This is always a very popular talk at VMworld, so come early!

VMworld Europe - Gran Fira - Barcelona

Monday, 11thSeptember 2017

[VIRT1971QE] Virtualizing Big Data: Quick Talk: 1:00-1:30pm

Gives an overview of other technical sessions that will be held in the main conference breakout sessions.

Tuesday,12thSeptember2017

[VIRT1351BE] New Architectures for Virtualizing Spark and Big Data Workloads Virtualizing Big Data: Quick Talk: 2:00-3:00pm

This talk looks into the viability of using different file systems to the Hadoop Distributed File System (HDFS) and explores a set ofalternative storage mechanisms for that data. These are becoming more common as big data matures on virtualization technology.

[MMC3164BE] How Data Science is Transforming Operations: Introduction to Wavefront byVMware: 2:00-3:00pm

Companies are re-thinking how they monitor and manage their systems, The data science techniques that WaveFront customers like Box, Lyft, Workday and Groupon use will be explored in this talk.

Wednesday,13thSeptember2017

[VIRT1445BE] Extreme Performance Series: Fast Virtualized Hadoop and Spark on All-Flash Disks: 5:00-6:00pm

Using vSphere 6.5 and the latest Intel hardware, VMware’s performance engineering lead on big data shows further evidence of the high performance and flexibility you can achieve by virtualizing your big data deployments. This is always a very popular talk at VMworld, so do sign upearly for it!

Thursday,14thSeptember2017

[FUT2020BE] Wringing Maximum Performance from vSphere for Extremely Demanding Workloads and Customers: 10:30-11:30am

In this Office of the CTO (VMware) session, the speakerswill share details of work theyhave done in three extremely performance-critical situations. They will talk about tuning and configuring VMWare ESXi to run a very latency-sensitive financial application, for example.

The post Big Data Technical Breakout Sessions and Related Events at VMworld 2017 appeared first on VMware vSphere Blog.

Read more..

Welcome to vSphere Central!

I am pleased to announce the release of a brand new technical resource center called vSphere Central. This site brings together a variety of technical assets from VMware that helps users understand how vSphere works, how to implement various features, plan upgrades, and more. With content in a range of formats, including text, video, and rich media tools such as walkthrough demos, the goal is to be a one-stop shop for original and curated technical content on vSphere.

A highlight of the site in this first release is thevSphere 6.5 Topology and Upgrade Planning Tool. I encourage you to read more about this tool if you are planning to upgrade to vSphere 6.5 (and if you haven’t already, it’s the perfect time now that vSphere 6.5 Update 1 has been released).

We plan to add more content to this site over time, so please make sure to bookmark vspherecentral.vmware.com, and let us know what you’d like to see.

The post Welcome to vSphere Central! appeared first on VMware vSphere Blog.

Read more..

Learn about vSphere 6.5 at VMworld

Ever since VMware released vSphere 6.5 late last year, the response from customers has been overwhelming positive. Customers have been especially excited by features such as enterprise-grade vCenter Server Appliance (VCSA), built-in encryption features, and a modern API set. Now that people have learned about these new capabilities, we decided for this year’s VMworld to present breakout sessions that focus more on the operation aspects of vSphere 6.5, such as architecture, deployment, and planning your upgrade. Of course, we still have a wide range of sessions covering such staple topics such as performance, troubleshooting, operations management, the new HTML5-based vSphere Client, PowerCLI, and more.

To help you plan your VMworld, we’ve put together a page that organizes all vSphere related sessions into topic areas and orders them by technical level. By consulting the VMworld 2017 vSphere course catalog, you’ll be able to find the sessions that are of interest to you and add them to your schedule. The list also includes Group Discussions, which are a great way to learn how your peers are using vSphere and provide feedback directly to product developers. We’ve also include both VMworld US and VMworld Europe sessions in the same list.

For quick reference, you can find this catalog at blogs.vmware.com/vsphere/vmworld.

We look forward to seeing you, whether in Las Vegas or Barcelona!

The post Learn about vSphere 6.5 at VMworld appeared first on VMware vSphere Blog.

Read more..

Kaminario K2 content pack for vRealize Log Insight

Overview

Kaminario K2 all-flash array delivers multi-petabyte scale storage with the performance and agility to meet the needs of an on-demand world. Kaminario K2 Gen6 is powered by VisionOSTM, a software platform that optimizes best-of-breed commodity hardware resources. K2 is built for delivering applications on highly scalable virtual environments, private clouds or as-a-service infrastructures.

K2&#rsquo;s VisionOS™ includes an open API platform – DataManage – that facilitates a tight integration with VMware vRealize Log Insight.

 

Dashboards, Fields, and Alerts provided by the Kaminario K2 Content Pack put you one click away from in depth analytics and event analysis. This content pack allows you to monitor and analyze Kaminario K2 all-flash arrays by converting the arrays&#rsquo; syslog messages into organized, pre-defined dashboard widgets.
General overview of multiple K2 all-flash arrays

 

Analyze events according to labels, severity and types

 

Audit login attempts to the K2

 

Minimum Requirements:
Kaminario K2 versions 6.0.2 or later
vRealize Log Insight 4.0 or later

Where do I get the Kaminario – K2 Content Pack?

You can download this Kaminario – K2 content pack from solution exchange –

https://marketplace.vmware.com/vsx/solutions/kaminario-k2-content-pack-for-vrealize-log-insight-1-0

OR You can download the content pack from the in-product Marketplace in Log Insight 2.5 or newer.

The post Kaminario K2 content pack for vRealize Log Insight appeared first on VMware Cloud Management.

Read more..

VMware vRealize Operations 6.6: Plan, Manage and Scale SDDC and Multi-Cloud Environments

Sneak Preview: What’s New in VMware vRealize Operations 6.6

Today, VMware announced upcoming releases for its entire portfolio of operations management products – vRealize Operations 6.6, vRealize Log Insight 4.5, vRealize Business for Cloud 7.3 and vRealize Network Insight 3.4.

The new vRealize Operations 6.6 release will offer an integrated experience with vRealize Log Insight and vRealize Business for Cloud, delivering the complete &#rsquo;intelligent operations&#rdquo; use case. This release will deliver several game-changing innovations, redesigned user experience, and greater/deeper integration across software defined datacenter (SDDC) technologies and multiple public clouds.

Intelligent operations solution is designed to help customers plan, manage and scale their software defined data centers (SDDC) and multi-cloud environments. If you&#rsquo;d like to be notified when the new vRealize Operations 6.6 is available to download/evaluate, sign up here. For more information visit the vRealize Operations web page.

vRealize Operations 6.6 will bring together all management functions – performance management, capacity, log analytics, cost analytics, planning, topology analysis, troubleshooting and automated workload balancing – in one integrated, highly intuitive, scalable and extensible platform.

VMware vRealize Operations 6.6 integrated with vRealize Log Insight 4.5 and vRealize Business for Cloud 7.3 delivers core capabilities around:

  • Application-aware SDDC and multi-cloud monitoring to help customers accelerate adoption of SDDC and integrate public clouds. Native SDDC integrations such as with VMware vSAN and VMware Cloud Foundation, redesigned intuitive user interface, unified visibility from applications to infrastructure and actionable insights combining metrics and logs deliver quick time to value. Customers get a unified operations view into applications and infrastructure health, visualize key performance indicators and infrastructure components dependencies. Predictive analytics and smart alerts enable proactive remediation of performance problems. Simple actionable out-of-the-box persona-based dashboards, metrics and logs side-by-side, custom dashboards, reports and views enable role-based access and smarter troubleshooting.
  • Automated and proactive performance management helps customers simplify operations, avoid disruption and free up time for more strategic tasks. The new capabilities in vRealize Operations 6.6. include fully automated workload balancing across hosts, clusters and datastores supporting both VMotion and Storage VMotion. It also offers full Distributed Resource Scheduler (DRS) management and predictive DRS. Predictive DRS combines predictive analytics from vRealize Operations with DRS capabilities to inform DRS of an expected spike in load so that DRS can move VMs before contention occurs. Anomalies and issues can be proactively detected and automatically remediated before end users are impacted.
  • Cloud Planning, capacity optimization and compliance – Upcoming vRealize Operations 6.6 includes tight integration with vRealize Business for Cloud which appears as &#rsquo;Business Management&#rdquo; tab within the vRealize Operations UI. Now vRealize Operations can correlate operational metrics with cost insights to understand how capacity utilization drives cost optimization. Fine grained cost analysis for private cloud and ability to compare costs across private and multiple public clouds such as AWS and Azure accelerate cloud planning, budgeting and procurement decisions, control costs and reduce risk. Customers can optimize cost and resource usage through capacity management, reclamation and right sizing and improve planning and forecasting. Also included are new SDDC health dashboards and hardening across the entire SDDC stack including NSX and VSAN.

 

 

 

Below are some of the &#rsquo;spotlight&#rdquo; features and capabilities:

1. Redesigned User Interface

  • Simplified Usability – New HTML5 UI has easier and consistent experience across all operations products
    • Quicker Navigation – New &#rsquo;Getting Started&#rdquo; dashboard points you to the right place
    • Accelerated Remediation – Persona-based dashboards provide the answers in one place – Operations, Troubleshooting, Cloud Capacity, Compliance and Config
  • Quicker Time To Value – OOTB integrations
    • Storage (vSAN), Logs (vRealize Log Insight), Business (vRealize Business for Cloud) and Automation (vRealize Automation)

 

 

 

 

 

 

 

 

 

 

 

2. Best for SDDC Management

  • Complete native vSAN management
    • Performance, Capacity, Logs, Configuration and Health
    • No management pack required, 1-click to enable rich VSAN metrics/data
  • Centralized management across vSAN stretched clusters
  • Deploy vSAN with confidence with complete visibility down to disk level
  • Operationalize vSAN with performance and capacity monitoring, including dedup and compression benefits
  • SDDC Health Dashboard, including vSphere, NSX and vSAN
    • SDDC-App-Operations-View with Health breakdown for each product
    • Correlation of SDDC-App-component to underlying Infrastructure health (deployment instances and corresponding VM health)
  • Service Discovery Management Pack
    • Discover and list services and display the relationship between selected service and other services on other VMs.
    • Visualize relationship of selected VM with other VMs
    • Service Breakdown Across Environment
    • Validate SRM protection group and recovery plan membership

 

 

3. Intelligent Workload Placement featuring Fully Automated Workload Balancing

  • Ensure performance across the datacenter with fully automated workload balancing
    • Across clusters
    • Across data stores (w/ storage vMotion)
    • Easily automatable
    • Schedulable at a convenient maintenance window
  • Ensure DRS Configurations – View and set DRS settings to drive better balance
  • Avoid contention – Predictive DRS takes action before the contention occurs
  • Proper Initial Placement - Utilize Operations analytics to optimize placement of workloads through vRA

 

4. Full 360-Degree Log Integration with vRealize Operations 6.6

  • Full Integration in-context within vRealize Operations for faster troubleshooting
    • Direct launch into Log insight Dashboard
    • Direct launch into Log Insight Interactive Analytics mode
    • Object auto-initiated log management
    • vRealize Operations alerts auto-initiated log management

 

5. Cost Management Gets Easier with vRealize Operations 6.6

  • Optimize Public Clouds Spending with more visibility
    • Cost Visibility: Costing analysis for all services on AWS and Azure
    • Usage Visibility: VM level usage visibility into public clouds
  • Enhanced planning by correlating capacity and costs
    • Fine grained cost analysis for private cloud
    • Makes capacity optimization quantifiable
    • Procurement planning

6. OOTB Compliance and Hardening

  • Tackle compliance problems
    • What is out of compliance
    • New vSphere hardening dashboard
    • Hardening across all SDDC components, vSAN, NSX, vCenter
  • Extend Compliance – PCI & HIPAA compliance for vSphere
  • Ensure business configurations - New cluster, host and VM configuration dashboards

 

 

 

 

 

 

 

 

 

 

For more information visit the vRealize Operations web page.

Again, if you&#rsquo;d like to be notified when the new vRealize Operations 6.6, vRealize Log Insight 4.5 and vRealize Business for Cloud 7.3 and vRealize Network Insight 3.4 become available to download/evaluate, sign up here.

 

The post VMware vRealize Operations 6.6: Plan, Manage and Scale SDDC and Multi-Cloud Environments appeared first on VMware Cloud Management.

Read more..

Tutto quello che dovete sapere su vSAN 6.6

Siamo felici di presentarvi la nuova generazione di VMware vSAN, la versione 6.6. Questa release estende ulteriormente il nostro posizionamento quale leader del HCI (Hyper-Converged Infrastructure), ambito che ci vede aiutare i clienti a modernizzare le loro infrastrutture più velocemente e in modo più semplice.

Quali sono le novità?

VMware vSAN è stata innovata in modo rapido e continuo. Ogni nuova versione ha visto VMware offrire un valore maggiore ai propri clienti, grazie alle nuove caratteristiche che hanno permesso loro di affrontare qualsiasi carico di lavoro e qualsiasi applicazione.

Più di 7.000 clienti hanno già scelto di affidare i propri carichi di lavoro più importanti a vSAN!
Questo momento di slancio non si è ancora arrestato, come confermano i 3 premi di settore che vSAN si è recentemente conquistata come prodotto dell’anno nel suo ambito.

La nuova versione è feature-packed, dotata di funzionalità enterprise avanzate, che rispondono alle tre esigenze chiave dei clienti: la sicurezza, la riduzione dei costi e l&#rsquo;ottimizzazione delle prestazioni.

Perché vSAN 6.6?

vSAN 6.6 si focalizza sulla capacità di offrire una soluzione HCI più economica, veloce e sicura, che garantisca ai clienti tre vantaggi principali:

  • Protezione avanzata – una sicurezza nativa hyper-converged infrastructure flessibile e a basso costo. vSAN Encryption consente ai clienti di crittografare i data-at-rest con una soluzione indipendente dal punto di vista dell’hardware che può funzionare su qualsiasi dispositivo di storage vSAN garantendo una gestione semplificata.
  • Riduzione del TCO - vSAN 6.6 diminuisce le pressioni in termini di budget IT riducendo il total cost of ownership fino al 50%. Le nuove funzionalità aiutano a ridurre le attività di gestione fino all&#rsquo;80% con controlli dell&#rsquo;installato intelligenti, l&#rsquo;hardware lifecycle management 1-click e analitiche avanzate basate su cloud.
  • Business continuity garantita - nuovi cluster potenziati e avanzati migliorano la business continuity sia nella protezione dei siti che del locale. Se un sito smette di funzionare, un secondo può svolgere interamente le sue funzioni senza interruzioni mantenendo, allo stesso tempo, un secondo layer di protezione rispetto a nodi locali o ai malfunzionamenti dei drive.
  • Prestazioni all-flash migliorate fino al 50% - i clienti possono ottenere una scalabilità all&#rsquo;avanguardia grazie alle maggiori prestazioni in vSAN 6.6 e all&#rsquo;hardware di nuova generazione, che aiutano i clienti a sostenere carichi di lavoro sempre maggiori.

Come vSAN sta aiutando i nostri clienti

Prendetevi qualche minuto per guardare i video qui sotto, potrete scoprire nel dettaglio i benefici che i nostri clienti hanno ottenuto dopo aver adottato vSAN.

 

 

Read more..

Performance of Storage I/O Control (SIOC) with SSD Datastores – vSphere 6.5

With Storage I/O Control (SIOC), vSphere 6.5 administrators can adjust the storage performance of VMs so that VMs with critical workloads will get the I/Os per second (IOPS) they need. Admins assign shares (the proportion of IOPS allocated to the VM), limits (the upper bound of VM IOPS), and reservations (the lower bound of VM IOPS) to the VMs whose IOPS need to be controlled. After shares, limits, and reservations have been set, SIOC is automatically triggered to meet the desired policies for the VMs.

A recently published paper shows the performance of SIOC meets expectations and successfully controls the number of IOPS for VM workloads.

We tested SIOC under a variety of conditions using the storage micro-benchmark Flexible I/O Tester (FIO) on two vSphere 6.5 hosts each with 8 VMs (for a total of 16 VMs) which shared one VMFS datastore that was made up of SSDs from an EMC VNX5700 hybrid storage array. Our tests showed the following:

  • Users can combine the three performance features (share, limit, and reservation) together to give a detailed performance specification (policy) for each VM. We tested the sample policies, provided in a vSphere cloud VM, for high priority, medium priority, and low priority VMs in our experiments to prove the efficiency of policy-based management.
  • In real datacenters, over a long period, storage usage is not stable. It is very normal to see sudden spikes in I/O requests amid a low-demand period. We mimicked the datacenter environment by introducing different types of VMs like logging, production, and others. We assigned each of the VMs a different priority level, and we tested the system performance over a long period to make sure that the overall system performance was
  • Uneven configuration is also very common among hosts. It is entirely possible that some hosts carry a heavier workload compared to others. We wanted to make sure that heavy computation demands on a host did not impact the I/O performance of the VMs located on that host. To mimic this condition, we created an uneven setting, using a shared datastore, and proved that the policy was still met, regardless of the number of VMs on each host.

To learn more about the performance experiments and test results, download Performance Implications of Storage I/O Control–Enabled SSD Datastores.

For instructions on how to use SIOC, see the vSphere 6.5 Resource Management guide > Chapter 8: Managing Storage I/O Resources.

The post Performance of Storage I/O Control (SIOC) with SSD Datastores - vSphere 6.5 appeared first on VMware VROOM! Blog.

Read more..

New Fling released – IOInsight

By Sankaran Sivathanu

VMware IOInsight is a tool to help people understand a VM’s storage I/O behavior. By understanding their VM’s I/O characteristics, customers can make better decisions about storage capacity planning and performance tuning. IOInsight ships as a virtual appliance that can be deployed in any vSphere environment and includes an intuitive web-based UI that allows users to choose VMDKs to monitor and view results.

Where does IOInsight help?

  • Customers may better tune and size their storage.
  • When contacting VMware Support for any vSphere storage issues, including a reportfromIOInsight can help VMware Support better understand the issues and can potentially lead to faster resolutions.
  • VMware Engineering can optimize products with a better understanding of various customers’ application behavior.

IOInsight captures I/O traces from ESXi and generates various aggregated metrics that represent the I/O behavior. The IOInsight report contains only these aggregated metrics and there is no sensitive information about the application itself. In addition to the built-in metrics computed by IOInsight, users can also write new analyzer plugins to IOInsight and visualize the results. A comprehensive SDK and development guide is included in the download bundle.

The fling works with vSphere 5.5 or above and can be downloaded athttps://labs.vmware.com/flings/ioinsight.

The post New Fling released – IOInsight appeared first on VMware VROOM! Blog.

Read more..

A look at All Paths Down in vSphere

Today we have a guest post from Karthick Sivaramakrishnan, who is a 3 year veteran at VMware. His primary field of expertise is vSphere Storage and Site Recovery Manager.

This blog post is centered around how ESXi handles unscheduled storage disconnects on vSphere 5.x and 6.x. An unscheduled storage disconnect means some issue in the vSphere environment has led to All-Paths-Down (APD) for a datastore. An APD situation will be seen when ESXi host does not have any path tocommunicate with a lun on the storage array.

ESXi host can encounter an APD under several conditions. As a result, we may end up having VMs running on a given datastore go down, the host could get disconnected from vCenter, and in worst cases ESXi could become unresponsive.

From vSphere version 5.x and onwards, we are able to discern whether a disconnect is permanent ortransient. Ideally a transient disconnect leads to All Paths Down state and ESXi expects the device to have a temporary disconnect. When we see permanent device loss or PDL the device is not expected to have a non-recoverable issue like a hardware error or the lun is unmapped.

In the below example we see all iSCSI datastores are in inactive state.

To determine what caused this issue we see ESXi logs, particularly vmkernel and vobd. This issue will be evident in the vmkernel logs.

vmkernel log

2017-01-10T13:04:26.803Z cpu1:32896)StorageApdHandlerEv: 110: Device or filesystem with identifier [naa.6000eb31dffdc33a0000000000000028] has entered the All Paths Down state.

2017-01-10T13:04:26.818Z cpu0:32896)StorageApdHandlerEv: 110: Device or filesystem with identifier [naa.6000eb31dffdc33a000000000000002a] has entered the All Paths Down state.

vobd log

2017-01-10T13:04:26.905Z: [scsiCorrelator] 475204262us: [esx.problem.storage.connectivity.lost] Lost connectivity to storage device naa.6000eb31dffdc33a0000000000000028. Path vmhba33:C0:T1:L0 is down. Affected datastores: “Green”.

2017-01-10T13:04:26.905Z: [scsiCorrelator] 475204695us: [esx.problem.storage.connectivity.lost] Lost connectivity to storage device naa.6000eb31dffdc33a000000000000002a. Path vmhba33:C0:T0:L0 is down. Affected datastores: “Grey”.

From these logs we understand that ESXi host has lost connectivity to the datastore. Any virtual machines using the affected datastore may become unresponsive. In this example while the datastores was mounted on ESXi, we lost the network uplink on the nic that was used for iSCSI connection. This was a transient issue and the datastore came up once the network uplink was restored.

In the below example we see Datastore Black is in inactive state.

If we look into the logs to determine whats going on we see these events.

Vmkernel.log

2017-01-09T12:42:09.365Z cpu0:32888)ScsiDevice: 6878: Device naa.6000eb31dffdc33a0000000000000063 APD Notify PERM LOSS; token num:1

2017-01-09T12:42:09.366Z cpu1:32916)StorageApdHandler: 1066: Freeing APD handle 0x430180b88880 [naa.6000eb31dffdc33a0000000000000063]

2017-01-09T12:49:01.260Z cpu1:32786)WARNING: NMP: nmp_PathDetermineFailure:2973: Cmd (0xc1) PDL error (0x5/0x25/0x0) - path vmhba33:C0:T3:L0 device naa.6000eb31dffdc33a0000000000000063 - triggering path evaluation

2017-01-09T12:49:01.260Z cpu1:32786)ScsiDeviceIO: 2651: Cmd(0x439d802ec580) 0xfe, CmdSN 0x4b7 from world 32776 to dev “naa.6000eb31dffdc33a0000000000000063” failed H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x25 0x0.

2017-01-09T12:49:01.300Z cpu0:40210)WARNING: NMP: vmk_NmpSatpIssueTUR:1043: Device naa.6000eb31dffdc33a0000000000000063 path vmhba33:C0:T3:L0 has been unmapped from the array

After some time passes you will see this message:

2017-01-09T13:13:11.942Z cpu0:32872)ScsiDevice: 1718: Permanently inaccessible device :naa.6000eb31dffdc33a0000000000000063 has no more open connections. It is now safe to unmount datastores (if any) and delete the device.

In this case the lun was unmapped from the array for this host and that is not a transient issue. Sens data 0x5 0x25 0x0 corresponds to &#rsquo;LOGICAL UNIT NOT SUPPORTED&#rdquo; which indicates the device is in Permanent Device Loss (PDL) state. Once ESXi knows the device is in PDL state it does not wait for the device to return back.

ESXi only checks ASC/ASCQ and if it happens to be 0x25/0x0 or 0x68/0x0, it marks device as PDL.

VMware KB 2004684 has in-depth information around APD and PDL situations. It also talks about planned and unplanned PDL. You can read it here:Permanent Device Loss (PDL) and All-Paths-Down (APD) in vSphere 5.x and 6.x (2004684)

Further on in the hostd logs you will see some additional events that will correlate to storage connection. Look for the below event id&#rsquo;s.

Event ID : esx.problem.storage.connectivity.lost

&#rsquo;esx.problem.storage.connectivity.lost&#rdquo; event indicates a loss in connectivity to the specified storage device. Any virtual machines using the affected datastore may become unresponsive.

Event ID : esx.problem.scsi.device.state.permanentloss

&#rsquo;esx.problem.scsi.device.state.permanentloss&#rdquo; event indicates a permanent device loss.

The post A look at All Paths Down in vSphere appeared first on Support Insider.

Read more..

Go Que Newsroom Categories

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 53 bytes)
in /home/content/36/8658336/html/goquecom/wp-includes/wp-db.php on line 2022

Query Monitor