Skip to content

Commit 0ffb727

Browse files
committed
Only use metadata patterns for generic types; perform other
initialization in-place on demand. Initialize parent metadata references correctly on struct and enum metadata. Also includes several minor improvements related to relative pointers that I was using before deciding to simply switch the parent reference to an absolute reference to get better access patterns. Includes a fix since the earlier commit to make enum metadata writable if they have an unfilled payload size. This didn't show up on Darwin because "constant" is currently unenforced there in global data containing relocations. This patch requires an associated LLDB change which is being submitted in parallel.
1 parent 790b7de commit 0ffb727

34 files changed

+1531
-619
lines changed

include/swift/Basic/RelativePointer.h

Lines changed: 117 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,114 @@
1717
// unnecessary relocation at dynamic linking time. This header contains types
1818
// to help dereference these relative addresses.
1919
//
20+
// Theory of references to objects
21+
// -------------------------------
22+
//
23+
// A reference can be absolute or relative:
24+
//
25+
// - An absolute reference is a pointer to the object.
26+
//
27+
// - A relative reference is a (signed) offset from the address of the
28+
// reference to the address of its direct referent.
29+
//
30+
// A relative reference can be direct, indirect, or symbolic.
31+
//
32+
// In a direct reference, the direct referent is simply the target object.
33+
// Generally, a statically-emitted relative reference can only be direct
34+
// if it can be resolved to a constant offset by the linker, because loaders
35+
// do not support forming relative references. This means that either the
36+
// reference and object must lie within the same linkage unit or the
37+
// difference must be computed at runtime by code.
38+
//
39+
// In a symbolic reference, the direct referent is a string holding the symbol
40+
// name of the object. A relative reference can only be symbolic if the
41+
// object actually has a symbol at runtime, which may require exporting
42+
// many internal symbols that would otherwise be strippable.
43+
//
44+
// In an indirect reference, the direct referent is a variable holding an
45+
// absolute reference to the object. An indirect relative reference may
46+
// refer to an arbitrary symbol, be it anonymous within the linkage unit
47+
// or completely external to it, but it requires the introduction of an
48+
// intermediate absolute reference that requires load-time initialization.
49+
// However, this initialization can be shared among all indirect references
50+
// within the linkage unit, and the linker will generally place all such
51+
// references adjacent to one another to improve load-time locality.
52+
//
53+
// A reference can be made a dynamic union of more than one of these options.
54+
// This allows the compiler/linker to use a direct reference when possible
55+
// and a less-efficient option where required. However, it also requires
56+
// the cases to be dynamically distinguished. This can be done by setting
57+
// a low bit of the offset, as long as the difference between the direct
58+
// referent's address and the reference is a multiple of 2. This works well
59+
// for "indirectable" references because most objects are known to be
60+
// well-aligned, and the cases that aren't (chiefly functions and strings)
61+
// rarely need the flexibility of this kind of reference. It does not
62+
// work quite as well for "possibly symbolic" references because C strings
63+
// are not naturally aligned, and making them aligned generally requires
64+
// moving them out of the linker's ordinary string section; however, it's
65+
// still workable.
66+
//
67+
// Finally, a relative reference can be near or far. A near reference
68+
// is potentially smaller, but it requires the direct referent to lie
69+
// within a certain distance of the reference, even if dynamically
70+
// initialized.
71+
//
72+
// In Swift, we always prefer to use a near direct relative reference
73+
// when it is possible to do so: that is, when the relationship is always
74+
// between two global objects emitted in the same linkage unit, and there
75+
// is no compatibility constraint requiring the use of an absolute reference.
76+
//
77+
// When more flexibility is required, there are several options:
78+
//
79+
// 1. Use an absolute reference. Size penalty on 64-bit. Requires
80+
// load-time work.
81+
//
82+
// 2. Use a far direct relative reference. Size penalty on 64-bit.
83+
// Requires load-time work when object is outside linkage unit.
84+
// Generally not directly supported by loaders.
85+
//
86+
// 3. Use an always-indirect relative reference. Size penalty of one
87+
// pointer (shared). Requires load-time work even when object is
88+
// within linkage unit.
89+
//
90+
// 4. Use a near indirectable relative reference. Size penalty of one
91+
// pointer (shared) when reference exceeds range. Runtime / code-size
92+
// penalty on access. Requires load-time work (shared) only when
93+
// object is outside linkage unit.
94+
//
95+
// 5. Use a far indirectable relative reference. Size penalty on 64-bit.
96+
// Size penalty of one pointer (shared) when reference exceeds range
97+
// and is initialized statically. Runtime / code-size penalty on access.
98+
// Requires load-time work (shared) only when object is outside linkage
99+
// unit.
100+
//
101+
// 6. Use a near or far symbolic relative reference. No load-time work.
102+
// Severe runtime penalty on access. Requires custom logic to statically
103+
// optimize. Requires emission of symbol for target even if private
104+
// to linkage unit.
105+
//
106+
// 7. Use a near or far direct-or-symbolic relative reference. No
107+
// load-time work. Severe runtime penalty on access if object is
108+
// outside of linkage unit. Requires custom logic to statically optimize.
109+
//
110+
// In general, it's our preference in Swift to use option #4 when there
111+
// is no possibility of initializing the reference dynamically and option #5
112+
// when there is. This is because it is infeasible to actually share the
113+
// memory for the intermediate absolute reference when it must be allocated
114+
// dynamically.
115+
//
116+
// Symbolic references are an interesting idea that we have not yet made
117+
// use of. They may be acceptable in reflective metadata cases where it
118+
// is desireable to heavily bias towards never using the metadata. However,
119+
// they're only profitable if there wasn't any other indirect reference
120+
// to the target, and it is likely that their optimal use requires a more
121+
// intelligent toolchain from top to bottom.
122+
//
123+
// Note that the cost of load-time work also includes a binary-size penalty
124+
// to store the loader metadata necessary to perform that work. Therefore
125+
// it is better to avoid it even when there are dynamic optimizations in
126+
// place to skip the work itself.
127+
//
20128
//===----------------------------------------------------------------------===//
21129

22130
#ifndef SWIFT_BASIC_RELATIVEPOINTER_H
@@ -279,7 +387,8 @@ class RelativeDirectPointer<RetTy (ArgTy...), Nullable, Offset> :
279387

280388
/// A direct relative reference to an aligned object, with an additional
281389
/// tiny integer value crammed into its low bits.
282-
template<typename PointeeTy, typename IntTy, typename Offset = int32_t>
390+
template<typename PointeeTy, typename IntTy, bool Nullable = false,
391+
typename Offset = int32_t>
283392
class RelativeDirectPointerIntPair {
284393
Offset RelativeOffsetPlusInt;
285394

@@ -305,9 +414,14 @@ class RelativeDirectPointerIntPair {
305414
using PointerTy = PointeeTy*;
306415

307416
PointerTy getPointer() const & {
417+
Offset offset = (RelativeOffsetPlusInt & ~getMask());
418+
419+
// Check for null.
420+
if (Nullable && offset == 0)
421+
return nullptr;
422+
308423
// The value is addressed relative to `this`.
309-
uintptr_t absolute = detail::applyRelativeOffset(this,
310-
RelativeOffsetPlusInt & ~getMask());
424+
uintptr_t absolute = detail::applyRelativeOffset(this, offset);
311425
return reinterpret_cast<PointerTy>(absolute);
312426
}
313427

include/swift/Runtime/Metadata.h

Lines changed: 41 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -34,31 +34,6 @@
3434

3535
namespace swift {
3636

37-
/// A bump pointer for metadata allocations. Since metadata is (currently)
38-
/// never released, it does not support deallocation. This allocator by itself
39-
/// is not thread-safe; in concurrent uses, allocations must be guarded by
40-
/// a lock, such as the per-metadata-cache lock used to guard metadata
41-
/// instantiations. All allocations are pointer-aligned.
42-
class MetadataAllocator {
43-
/// Address of the next available space. The allocator grabs a page at a time,
44-
/// so the need for a new page can be determined by page alignment.
45-
///
46-
/// Initializing to -1 instead of nullptr ensures that the first allocation
47-
/// triggers a page allocation since it will always span a "page" boundary.
48-
char *next = (char*)(~(uintptr_t)0U);
49-
50-
public:
51-
MetadataAllocator() = default;
52-
53-
// Don't copy or move, please.
54-
MetadataAllocator(const MetadataAllocator &) = delete;
55-
MetadataAllocator(MetadataAllocator &&) = delete;
56-
MetadataAllocator &operator=(const MetadataAllocator &) = delete;
57-
MetadataAllocator &operator=(MetadataAllocator &&) = delete;
58-
59-
void *alloc(size_t size);
60-
};
61-
6237
template <unsigned PointerSize>
6338
struct RuntimeTarget;
6439

@@ -90,6 +65,10 @@ struct InProcess {
9065

9166
template <typename T, bool Nullable = false>
9267
using FarRelativeDirectPointer = FarRelativeDirectPointer<T, Nullable>;
68+
69+
template <typename T, bool Nullable = false>
70+
using FarRelativeIndirectablePointer =
71+
FarRelativeIndirectablePointer<T, Nullable>;
9372

9473
template <typename T, bool Nullable = true>
9574
using RelativeDirectPointer = RelativeDirectPointer<T, Nullable>;
@@ -118,6 +97,9 @@ struct External {
11897

11998
template <typename T, bool Nullable = false>
12099
using FarRelativeDirectPointer = StoredPointer;
100+
101+
template <typename T, bool Nullable = false>
102+
using FarRelativeIndirectablePointer = StoredSize;
121103

122104
template <typename T, bool Nullable = true>
123105
using RelativeDirectPointer = int32_t;
@@ -145,6 +127,10 @@ template <typename Runtime, typename Pointee, bool Nullable = true>
145127
using TargetRelativeDirectPointer
146128
= typename Runtime::template RelativeDirectPointer<Pointee, Nullable>;
147129

130+
template <typename Runtime, typename Pointee, bool Nullable = true>
131+
using TargetFarRelativeIndirectablePointer
132+
= typename Runtime::template FarRelativeIndirectablePointer<Pointee,Nullable>;
133+
148134
struct HeapObject;
149135

150136
template <typename Runtime> struct TargetMetadata;
@@ -1480,16 +1466,33 @@ struct TargetNominalTypeDescriptor {
14801466
};
14811467

14821468
RelativeDirectPointerIntPair<TargetGenericMetadata<Runtime>,
1483-
NominalTypeKind>
1469+
NominalTypeKind, /*Nullable*/ true>
14841470
GenericMetadataPatternAndKind;
14851471

1472+
using NonGenericMetadataAccessFunction = const Metadata *();
1473+
1474+
/// A pointer to the metadata access function for this type.
1475+
///
1476+
/// The type of the returned function is speculative; in reality, it
1477+
/// takes one argument for each of the generic requirements, in the order
1478+
/// they are listed. Therefore, the function type is correct only if
1479+
/// this type is non-generic.
1480+
///
1481+
/// Not all type metadata have access functions.
1482+
TargetRelativeDirectPointer<Runtime, NonGenericMetadataAccessFunction,
1483+
/*nullable*/ true> AccessFunction;
1484+
14861485
/// A pointer to the generic metadata pattern that is used to instantiate
14871486
/// instances of this type. Zero if the type is not generic.
14881487
TargetGenericMetadata<Runtime> *getGenericMetadataPattern() const {
14891488
return const_cast<TargetGenericMetadata<Runtime>*>(
14901489
GenericMetadataPatternAndKind.getPointer());
14911490
}
14921491

1492+
NonGenericMetadataAccessFunction *getAccessFunction() const {
1493+
return AccessFunction.get();
1494+
}
1495+
14931496
NominalTypeKind getKind() const {
14941497
return GenericMetadataPatternAndKind.getInt();
14951498
}
@@ -1928,9 +1931,9 @@ struct TargetValueMetadata : public TargetMetadata<Runtime> {
19281931
Description;
19291932

19301933
/// The parent type of this member type, or null if this is not a
1931-
/// member type.
1932-
FarRelativeIndirectablePointer<const TargetMetadata<Runtime>,
1933-
/*nullable*/ true> Parent;
1934+
/// member type. It's acceptable to make this a direct pointer because
1935+
/// parent types are relatively uncommon.
1936+
TargetPointer<Runtime, const TargetMetadata<Runtime>> Parent;
19341937

19351938
static bool classof(const TargetMetadata<Runtime> *metadata) {
19361939
return metadata->getKind() == MetadataKind::Struct
@@ -2778,12 +2781,6 @@ struct TargetProtocolConformanceRecord {
27782781
using ProtocolConformanceRecord
27792782
= TargetProtocolConformanceRecord<InProcess>;
27802783

2781-
/// \brief Fetch a uniqued metadata object for a nominal type with
2782-
/// resilient layout.
2783-
SWIFT_RUNTIME_EXPORT
2784-
extern "C" const Metadata *
2785-
swift_getResilientMetadata(GenericMetadata *pattern);
2786-
27872784
/// \brief Fetch a uniqued metadata object for a generic nominal type.
27882785
///
27892786
/// The basic algorithm for fetching a metadata object is:
@@ -3028,11 +3025,16 @@ struct ClassFieldLayout {
30283025

30293026
/// Initialize the field offset vector for a dependent-layout class, using the
30303027
/// "Universal" layout strategy.
3028+
///
3029+
/// This will relocate the metadata if it doesn't have enough space
3030+
/// for its superclass. Note that swift_allocateGenericClassMetadata will
3031+
/// never produce a metadata that requires relocation.
30313032
SWIFT_RUNTIME_EXPORT
3032-
extern "C" void swift_initClassMetadata_UniversalStrategy(ClassMetadata *self,
3033-
size_t numFields,
3034-
const ClassFieldLayout *fieldLayouts,
3035-
size_t *fieldOffsets);
3033+
extern "C" ClassMetadata *
3034+
swift_initClassMetadata_UniversalStrategy(ClassMetadata *self,
3035+
size_t numFields,
3036+
const ClassFieldLayout *fieldLayouts,
3037+
size_t *fieldOffsets);
30363038

30373039
/// \brief Fetch a uniqued metadata for a metatype type.
30383040
SWIFT_RUNTIME_EXPORT

include/swift/Runtime/RuntimeFunctions.def

Lines changed: 5 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -568,12 +568,6 @@ FUNCTION(GetForeignTypeMetadata, swift_getForeignTypeMetadata, DefaultCC,
568568
ARGS(TypeMetadataPtrTy),
569569
ATTRS(NoUnwind, ReadNone)) // only writes to runtime-private fields
570570

571-
// Metadata *swift_getResilientMetadata(GenericMetadata *pattern);
572-
FUNCTION(GetResilientMetadata, swift_getResilientMetadata, DefaultCC,
573-
RETURNS(TypeMetadataPtrTy),
574-
ARGS(TypeMetadataPatternPtrTy),
575-
ATTRS(NoUnwind, ReadOnly))
576-
577571
// Metadata *swift_getGenericMetadata(GenericMetadata *pattern,
578572
// const void *arguments);
579573
FUNCTION(GetGenericMetadata, swift_getGenericMetadata, RegisterPreservingCC,
@@ -664,13 +658,13 @@ FUNCTION(GetExistentialMetadata,
664658
ATTRS(NoUnwind, ReadOnly))
665659

666660
// struct FieldInfo { size_t Size; size_t AlignMask; };
667-
// void swift_initClassMetadata_UniversalStrategy(Metadata *self,
668-
// size_t numFields,
669-
// const FieldInfo *fields,
670-
// size_t *fieldOffsets);
661+
// Metadata *swift_initClassMetadata_UniversalStrategy(Metadata *self,
662+
// size_t numFields,
663+
// const FieldInfo *fields,
664+
// size_t *fieldOffsets);
671665
FUNCTION(InitClassMetadataUniversal,
672666
swift_initClassMetadata_UniversalStrategy, DefaultCC,
673-
RETURNS(VoidTy),
667+
RETURNS(TypeMetadataPtrTy),
674668
ARGS(TypeMetadataPtrTy, SizeTy,
675669
SizeTy->getPointerTo(),
676670
SizeTy->getPointerTo()),
@@ -934,10 +928,6 @@ FUNCTION(RegisterTypeMetadataRecords,
934928
ARGS(TypeMetadataRecordPtrTy, TypeMetadataRecordPtrTy),
935929
ATTRS(NoUnwind))
936930

937-
FUNCTION(InitializeSuperclass, swift_initializeSuperclass, DefaultCC,
938-
RETURNS(VoidTy),
939-
ARGS(TypeMetadataPtrTy, Int1Ty),
940-
ATTRS(NoUnwind))
941931
FUNCTION(InstantiateObjCClass, swift_instantiateObjCClass, DefaultCC,
942932
RETURNS(VoidTy),
943933
ARGS(TypeMetadataPtrTy),

lib/IRGen/ClassMetadataLayout.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,7 @@ template <class Impl> class ClassMetadataLayout : public MetadataLayout<Impl> {
9595

9696
// Add a reference to the parent class, if applicable.
9797
if (theClass->getDeclContext()->isTypeContext()) {
98-
asImpl().addParentMetadataRef(theClass);
98+
asImpl().addParentMetadataRef(theClass, type);
9999
}
100100

101101
// Add space for the generic parameters, if applicable.
@@ -232,7 +232,7 @@ class ClassMetadataScanner : public ClassMetadataLayout<Impl> {
232232
void addIVarDestroyer() { addPointer(); }
233233
void addValueWitnessTable() { addPointer(); }
234234
void addDestructorFunction() { addPointer(); }
235-
void addParentMetadataRef(ClassDecl *forClass) { addPointer(); }
235+
void addParentMetadataRef(ClassDecl *forClass, Type classType) {addPointer();}
236236
void addSuperClass() { addPointer(); }
237237
void addClassFlags() { addInt32(); }
238238
void addInstanceAddressPoint() { addInt32(); }

0 commit comments

Comments
 (0)