前言:
openstack的部署和使用是难度比较大的,难免会出现各种各样的问题,因此,本文将把一些在部署和使用openstack社区版时出现的错误做一个记录,并就每一个错误分析和解决问题。(尽量记录比较经典的错误,太弱智的那些就不记录了)
一,
错误前情回顾:
创建虚拟机实例失败
创建实例的命令是:
openstack server create --flavor m1.tiny --image Centos7 --nic net-id=688a0356-4f2b-4029-b49e-a11bbdbedf0b --key-name mykey --security-group e5f115d2-3c65-4bc5-89db-b9f725ee81db centos7-init2
错误: 实例 "centos7-init2" 执行所请求操作失败,实例处于错误状态。: 请稍后再试 [错误: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 1bd5d1af-17b1-4524-9f9c-6be299d68293.].
在dashboard内,查看实例详情,可以看到错误日志:
消息
Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 17fe6f00-4560-476a-a79b-e6f62d866155.
编码
500
详情
Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 624, in build_instances raise exception.MaxRetriesExceeded(reason=msg) MaxRetriesExceeded: Exceeded maximum number of retries. Exhausted all hosts available for retrying build failures for instance 17fe6f00-4560-476a-a79b-e6f62d866155.
已创建
Feb. 11, 2023, 5:23 p.m.
虚拟机实例状态是:
解析:
错误代码是500,并且虚拟机没有获取到fix ip地址,上面最后一图IP地址那是空缺的
以上表明错误发生在网络层面,错误提示是在尝试N次后,虚拟机实例仍无法调度,因此,错误日志必定在nova这个服务内有所体现
因此,在所有包含nova的节点执行以下命令,快速查找错误:
grep "ERROR" /var/log/nova/*
在其中一个计算节点,发现如下日志:
倒数第二行提示security group没有被发现
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1042, in allocate_for_instance
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293] instance, neutron, security_groups)
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 829, in _process_security_groups
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293] security_group_id=security_group)
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293] SecurityGroupNotFound: Security group e5f115d2-3c65-4bc5-89db-b9f725ee81db not found.
/var/log/nova/nova-compute.log:2023-02-11 16:49:29.901 970 ERROR nova.compute.manager [instance: 1bd5d1af-17b1-4524-9f9c-6be299d68293]
结合实例启动的时间和日志文件内的上下文,可以确定原因是security group没有找到的原因
查看安全组:
[root@openstack1 ~]# openstack security group list
+--------------------------------------+---------+------------------------+----------------------------------+------+
| ID | Name | Description | Project | Tags |
+--------------------------------------+---------+------------------------+----------------------------------+------+
| 2035d43a-0e81-4257-bd23-13af431b9f91 | default | Default security group | 205ce8addd9444c893bd62244bcdae78 | [] |
| 5b1b71d7-5e24-4011-82ca-0ddabfd32e8a | default | Default security group | | [] |
| e5f115d2-3c65-4bc5-89db-b9f725ee81db | default | Default security group | ae2263d201c0437788c85f1178b91dbe | [] |
+--------------------------------------+---------+------------------------+----------------------------------+------+
[root@openstack1 ~]# openstack project list
+----------------------------------+---------+
| ID | Name |
+----------------------------------+---------+
| 205ce8addd9444c893bd62244bcdae78 | admin |
| ae2263d201c0437788c85f1178b91dbe | service |
+----------------------------------+---------+
这里就很奇怪了,安全组可以看到,但openstack创建实例的时候又找不到了,可能和前面调整了网络有关系,
解决方案:
要么重建安全组,要么使用第一个 也就是admin项目的安全组,因为是用的admin登陆的openstack
删除实例,重新创建,创建的时候使用2035d43a-0e81-4257-bd23-13af431b9f91 这个安全组:
[root@openstack1 ~]# openstack server delete centos7-init2
[root@openstack1 ~]# openstack server create --flavor m1.tiny --image Centos7 --nic net-id=688a0356-4f2b-4029-b49e-a11bbdbedf0b --key-name mykey --security-group 2035d43a-0e81-4257-bd23-13af431b9f91 --user-data /tmp/centos.config --config-drive true centos7-init2
+-------------------------------------+------------------------------------------------+
| Field | Value |
+-------------------------------------+------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | |
| OS-EXT-SRV-ATTR:host | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| OS-EXT-SRV-ATTR:instance_name | |
| OS-EXT-STS:power_state | NOSTATE |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | None |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| adminPass | YpnXPX8xmK7T |
| config_drive | True |
| created | 2023-02-11T09:18:31Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | 9d24d3d9-e42c-4267-816e-ad8414cd3c7f |
| image | Centos7 (2b672c8f-75bf-4fe0-8468-37309563025a) |
| key_name | mykey |
| name | centos7-init2 |
| progress | 0 |
| project_id | 205ce8addd9444c893bd62244bcdae78 |
| properties | |
| security_groups | name='2035d43a-0e81-4257-bd23-13af431b9f91' |
| status | BUILD |
| updated | 2023-02-11T09:18:31Z |
| user_id | 74bc206609e04092b698698d944e922a |
| volumes_attached | |
+-------------------------------------+------------------------------------------------+
再次查看实例状态,可以看到没有错误了。dashboard里也看不到错误了:
[root@openstack1 ~]# openstack server list
+--------------------------------------+---------------+---------+--------------------------+---------+---------+
| ID | Name | Status | Networks | Image | Flavor |
+--------------------------------------+---------------+---------+--------------------------+---------+---------+
| 9d24d3d9-e42c-4267-816e-ad8414cd3c7f | centos7-init2 | ACTIVE | provider=192.168.123.160 | Centos7 | m1.tiny |
二,
前情回顾:
No valid host was found.
还是创建虚拟机失败
错误代码仍然是500
消息
No valid host was found.
编码
500
详情
Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1271, in schedule_and_build_instances instance_uuids, return_alternates=True) File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 784, in _schedule_instances return_alternates=return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 50, in select_destinations instance_uuids, return_objects, return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 35, in __run_method return getattr(self.instance, __name)(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations instance_uuids, return_objects, return_alternates) File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 160, in select_destinations return cctxt.call(ctxt, 'select_destinations', **msg_args) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 179, in call retry=self.retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 133, in _send retry=retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 645, in send call_monitor_timeout, retry=retry) File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 636, in _send raise result NoValidHost_Remote: No valid host was found. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 154, in select_destinations raise exception.NoValidHost(reason="") NoValidHost: No valid host was found.
错误分析:
这个错误比较常见的,也没什么必要查看日志,基本都是系统资源不足造成的
解决方案:
删除不使用的多余的实例,在重新创建即可,或者是实例类型用错了导致资源不够。
例如:
[root@openstack1 ~]# openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 0 | m1.nano | 64 | 1 | 0 | 1 | True |
| 1 | m1.tiny | 1024 | 50 | 0 | 1 | True |
| 2 | m1.small | 2048 | 500 | 0 | 1 | True |
| 3 | m1.medium | 4096 | 500 | 0 | 2 | True |
| 4 | m1.large | 8192 | 500 | 0 | 4 | True |
| 5 | m1.xlarge | 16384 | 500 | 0 | 8 | True |
+----+-----------+-------+------+-----------+-------+-----------+
整个系统才100G硬盘,16G内存,创建实例的时候指定m1.xlarge 这个实例规格必定是创建不了并报以上错误的。